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ABSTRACT 


Non-developmental  Items  (NDI)  acquisition  programs  are  enjoying  popular 
support  as  faster,  cheaper  alternatives  to  full-scale  development  programs. 
Unfortunately,  DOD  policy  with  respect  to  risk  management  in  NDI  programs  is 
lacking.  Tailoring  DOD  risk  management  policy  to  support  NDI  program 
management  leaves  the  program  manager  (PM)  much  guess-work.  A  NDI  PM's 
risk  management  program  cannot  reasonably  benefit  from  DOD  risk  management 
guidance,  procedures,  and  risk  management  tools  because  they  are  oriented  to 
developmental  program  risks  and  risk  management  practices.  Missing  is  any 
explicit  consideration  of  the  unique  risks  and  risk  management  requirements  in 
NDI  programs.  NDI  PMs  need  more  explicit  guidance  in  policy  and  instruction 
regarding  NDI  risk  management  in  the  streamlined,  accelerated  NDI  environment. 
This  need  is  brought  out  in  a  case  study  of  the  Forward  Area  Air  Defense  Sensors 
Product  Office  which  attempts  to  implement  sound  risk  management  into  its  NDI 
products  without  the  benefit  of  definitive  NDI  risk  identification,  assessment,  or 
response  policy  material.  A  lesson  learned  is  the  need  for  a  published  Risk 
Management  Plan  as  the  source  of  NDI  risk  management  program  decisions  and 
actions.  Specific  recommendations  are  contained  for  inclusion  in  DOD  policy  with 
respect  to  NDI  risk  management. 
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I .   INTRODUCTION 
A.   BACKGROUND 

Today's  era  of  defense  downsizing  has  the  military- 
services  scrambling  to  protect  research  and  development  funds 
and  to  preserve  their  acquisition  programs.   Quality  of  life, 
training,  sustainment,  and  readiness  issues  continue  to  siphon 
funds  away  from  research,  development,  and  acquisition  (RD&A) 
programs.  The  Services  and  their  acquisition  program  managers 
(PMs)  are  hard-pressed  to  find  ways  to  reduce  development 
costs  and  program  duration  while  optimizing  new  system 
performance  and  support  capabilities.   In  these  lean  times, 
program  management  mistakes  and  problem-plagued  acquisition 
strategies  can  often  lead  to  program  cancellation.   Program 
risks  must  be  carefully  managed  to  prevent  unanticipated  or 
poorly  planned- for  events  from  making  a  program  vulnerable  to 
funding  cuts  or  elimination. 

To  the  rescue,  or  so  many  hope,  comes  the  Non- 
developmental  Item  (NDI)  .  NDI  system  acquisitions  give  PMs 
and  the  Government  greater  access  to  technology.  NDI 
acquisitions  benefit  industry  by  broadening  the  U.S.  Defense 
Technology  and  Industrial  Base  (DTIB) ,  increasing  the  number 
of  defense  contractors  available  to  offer  products  to  the 
Department  of  Defense  (DOD) ,  and  providing  more  extensive 
surge  capability  upon  mobilization  of  the  defense  industry 
(Gansler,  p.  110,  1994) .  Acquisition  strategies  based  on 
these  lower- cost,  shorter-duration  alternatives  to  the 
traditional,  full-scale  development  programs  offer  the  promise 
of  effective  risk  mitigation  and  program  success.  NDIs 
seemingly  offer  the  ideal,  low-risk  solution  of  state-of-the- 
art  technological  capabilities  that  can  meet  the  Services' 
urgent  mission  needs  and  operational  requirements. 

NDI  program  management,  however,  is  not  without  its  own 
risks.   A  PM  must  tailor  a  program's  risk  management  to  the 


unique  risks  and  uncertainties  in  a  NDI  system  acquisition. 
Tailoring  is  a  practice  that  means  modifying  the  acquisition 
process  as  needed  to  achieve  favorable  program  results  (DOD 
Instruction  (DODI)  5000.2,  p.  2-6,  1991).  The  concept  of 
tailoring  risk  management  is  valuable  as  part  of  the  PM'  s 
program  authority.  The  problems  arise  in  the  implementation 
of  tailored  risk  management  programs,  particularly  when 
considering  NDIs. 

Risk  management  in  NDI  acquisition  programs  requires  more 
than  tailoring.  It  demands  policy,  framework,  and  format  to 
assist  the  program  manager.  The  purpose  of  using  a  NDI 
acquisition  strategy  is  to  simplify  and  speed  the  acquisition 
process  to  meet  a  program's,  and  ultimately,  the  soldier's 
needs.  NDI  acquisition  is  an  approach  that  requires 
supportive  risk  management  policy  structure,  rather  than 
implicit  expectations  of  tailoring,  to  fulfill  its  promise  of 
simple,  rapid,  and  reduced- risk  acquisitions. 

B.   OBJECTIVE 

The  objective  of  this  thesis  is  to  examine  risk 
management  in  NDI  acquisition  programs.  It  will  consist  of 
DOD  risk  management  and  NDI  policy  reviews  from  an  Army  air 
defender's  point  of  view.  Risk  management  concepts  and 
practices  will  be  explored  first  to  set  the  stage  for  defense 
risk  management.  DOD  risk  management  policies  will  be 
assessed  to  determine  their  applicability  and  support  of  NDI 
risk  management  programs.  This  thesis  will  then  use  the 
Ground  Based  Sensor  (GBS)  and  the  Light  and  Special  Division 
Interim  Sensor  (LSDIS)  acquisitions  managed  by  the  Forward 
Area  Air  Defense  (FAAD)  Sensors  Program  Office  to  demonstrate 
how  risk  management  is  currently  practiced  by  an  Air  Defense 
Artillery  (ADA)  systems  program  office.  The  result  of  this 
examination  will  be  recommendations  for  risk  management 
policies  and  processes  focused  on  NDI  system  acquisitions. 


C.   RESEARCH  QUESTIONS 

The  primary  research  question  of  this  thesis  is: 
How  does  DOD  risk  management  policy  address  NDI  acquisitions 
and  how  does  it  support  NDI  acquisition  programs  such  the 
Forward  Area  Air  Defense  Early  Warning  Sensors  Program? 

Subsidiary  Research  Questions  supporting  this  issue  are: 

1.  What  is  risk  management? 

2 .  What  are  the  current  DOD  and  Army  risk  management 
policies  and  how  do  they  address  NDI  acquisitions? 


3.  What  are  the  risks  associated  with  NDI  acquisitions? 

4.  What  were  the  risks  involved  with  the  GBS  and  LSDIS 
tactical  early  warning  sensor  acquisitions? 

5 .  What  was  the  risk  management  approach  for  GBS  and 
LSDIS  and  how  consistent  was  the  FAAD  Sensor  program's 
risk  management  effort  in  complying  with  DOD  and  Army 
risk  management  policies?  What  problems  were 
encountered  in  complying  with  risk  management 
policies? 


6 .  What  improvements  can  be  made  to  DOD  risk  management 
policy  to  enhance  NDI  risk  management  planning  and 
execution  in  future  NDI  acquisitions? 

D.   SCOPE 

This  thesis  encompasses  risk  management  policy  as  it 
applies  to  NDI  acquisitions.  It  examines  the  current 
structure  of  risk  management  in  DOD  and  the  U.S.  Army.  It 
considers  the  unique  uncertainties  and  risk  in  NDI  programs. 
It  explores  whether  the  intent  of  using  NDI  acquisitions  as  a 
risk  management  measure  accomplishes  its  purpose.  It  focuses 
on  the  efforts  of  a  NDI  Air  Defense  system  program  management 
office  (PMO)  to  manage  its  program  and  its  program  risks  in  an 
era  of  scarce  resources  and  broad  risk  management  policy.  It 
is  not  meant  to  be  a  technical  examination  of  the  GBS  or  LSDIS 


systems  capabilities  nor  a  comprehensive  evaluation  of  their 
acquisition  strategies  or  plans.  It  will  explore  the 
treatment  of  aspects  of  risk  within  those  documents  but  will 
not  cover  classified  material.  This  research  does  not  contain 
material  from  the  Army's  risk  management  handbook  (Department 
of  the  Army  Pamphlet  70-2)  scheduled  to  be  published  in 
February  1995.  This  examination  assumes  a  working  knowledge 
of  a  system  acquisition  life  cycle. 

E.   LITERATURE  REVIEW  AND  METHODS 

The  literature  search  to  support  this  effort  is  driven  by 
first-hand  training  experiences  by  the  thesis  author.  The 
starting  point  for  this  analysis  comes  from  the  Defense 
Systems  Management  College  (DSMC)  Risk  Management,  NDI 
Acquisition,  Systems  Engineering,  and  Commercial  Practices 
publications,  ADA  Magazine  articles  on  GBS  and  LSDIS,  Program 
Manager  Magazine  articles  on  risk  management,  and  the  Air 
Force  Acquisition  Model  (AFAM)  materials  relating  to  risk 
management.  This  is  complemented  by  the  Program  Management 
Institute's  (PMI)  guidelines  for  commercial  program  risk 
management  which  provide  developed  risk  management  theory. 
Journal  articles  and  technical  reports  examining  risk 
management  and  NDI  acquisitions  were  obtained  from  the  Defense 
Logistics  Systems  Information  Exchange  (DLSIE)  and  the  Defense 
Technical  Information  Center  (DTIC) .  The  FAAD  Sensors  Program 
Office,  Assistant  Program  Executive  Office  for  ADA 
Integration,  and  the  Contracting  Directorate  at  Redstone 
Arsenal,  Alabama  provided  FAAD  Sensors  acquisition  and  risk 
management  documents  and  data.*  The  TRADOC  Systems  Manager 
(TSM)  for  Early  Warning  Sensors  and  the  Directorate  for  Combat 
Developments  (DCD)  at  Ft.  Bliss,  Texas  assisted  by  providing 
material  on  GBS  and  LSDIS  fielding. 

The  risk  management  materials  for  this  research  are 
augmented  with  face -to -face   interviews  with  experts  at 


Redstone  Arsenal  and  Fort  Bliss  working  with  GBS  and  LSDIS 
acquisitions.  Additional  data  came  from  telephone  discussions 
with  those  same  experts.  The  interviews  solicited  views  and 
opinions  on  the  GBS  and  LSDIS  NDI  acquisition  strategies, 
their  risk  management  programs,  issues  affecting  NDI 
acquisitions,  and  their  associated  risks.  Thesis  travel 
occurred  14-18  June  1994  to  conduct  the  personal  interviews 
and  discussions  with  these  sources. 

The  method  this  thesis  employs  is  to  familiarize  the 
reader  with  risk  management  and  NDI  concepts  and  the  DOD 
policies  pertaining  to  them.  It  includes  a  review  of  existing 
risk  management  policies  and  procedures  published  and  cited  by 
defense  sources.  It  progresses  to  a  specific  NDI  program  case 
analysis.  DOD  and  Army  risk  management  guidelines  are 
examined  through  the  case  study  of  two  ADA  NDI  acquisitions. 
The  case  reviews  how  DOD  policy  was  implemented  and  to  what 
extent  the  policy  supports  NDI  acquisition  program  managers. 
The  steps  used  to  research  and  analyze  this  thesis  topic 
consist  of  the  following: 

1.  A  familiarization  and  study  of  risk  management 
concepts  using  the  Program  Management  Institute's  body 
of  knowledge  pertaining  to  project  risk  management. 


2.  An  examination  of  defense  risk  management  and  risk 
management  policy.  It  is  complemented  by  the  DSMC 
publication  on  risk  management.  DOD  Acquisition 
policy  from  the  5000. -series  and  Army  risk  management 
regulations  provide  sources  for  this  step. 


3 .  An  assessment  of  the  support  and  effect  of  these 
policies  on  NDI  program  management  using  the  DSMC 
publication  on  NDI  acquistions  and  the  above-mentioned 
references . 


4.  An   assessment   of   tailoring   developmental   risk 
management  policy  in  support  of  NDI  acquisition  programs . 


An  analysis  of  a  case.  The  case  examines  the 
experience  of  the  FAAD  Sensors  PMO  using  DOD  risk 
management  policies  and  practices  in  its  program 
management  of  the  GBS  and  LSDIS.  The  basis  for  this 
case  is  face -to- face  and  telephone  interviews  with 
past  and  present  FAAD  Sensors  officials  using  a 
prepared  set  of  questions  on  the  FAAD  Sensors  program 
and  risk  management. 


Instruments  of  measurement  for  this  thesis  are 
templates  from  DOD  Publication  4245. 7-M,  the  risk 
assessment  formats  found  in  DOD  5000. 2 -M,  and  the  risk 
management  plan  cited  -in  the  DSMC  risk  management 
publication.  A  dendritic  diagram  will  depict  the 
differences  in  developmental  item  and  NDI  risk 
management  policy.  This  may  be  valuable  not  only  to 
the  GBS  and  LSDIS  case  but  for  NDI  acquisition 
programs  overall . 


Based  on  case  analysis  and  policy  assessment,  a  series 
of  recommendations  and  conclusions  emerge  to  aid 
future  NDI  PMs.  The  usefulness  of  NDI -based  risk 
templates  and  NDI -oriented  risk  management  policy  will 
be  included  as  the  basis  for  these  recommendations. 


F.  DEFINITIONS  AND  ACRONYMS 

Definitions  and  acronyms  common  to  DOD,  the  U.S.  Army, 
and  civilian  program  management  are  noted  throughout  this 
thesis. 

G.  CHAPTER  OUTLINE 

This  thesis  investigates  the  use  of  risk  management 
programs  and  policies  for  NDI  acquisitions. 

Chapter  I  introduces  the  background  and  focus  of 
research.  It  examines  today's  constrained  resources  which 
complicate  the  acquisition  and  environment  and  support  the 
appeal  of  NDI  solutions.  It  considers  the  incentives  for  risk 
management  and  the  unforgiving  acquisition  climate  in  which 


risk  management  has  become  a  must. 

Chapter  II  serves  as  an  overview  to  define  and  describe 
risk  and  risk  management  concepts .  This  risk  management 
"primer",  based  on  the  PMI  body  of  knowledge,  then  narrows  to 
consider  defense -unique  details  of  risk  management.  It 
introduces  current  risk  management  policy  at  both  DOD  and  Army 
level  and  lays  out  the  general  risk  management  process 
prescribed  for  program  offices. 

Chapter  III  examines  the  NDI  alternative  to  full  scale 
development.  It  discusses  the  acquisition  and  risk  management 
policy  that  is  expected  to  support  NDI  system  acquisition 
management.  It  includes  consideration  of  the  tailoring 
involved  in  NDI  risk  management.  It  examines  the  risks  unique 
to  pursuing  NDI  acquisitions. 

Chapter  IV  relates  the  events  that  created  the  current 
absence  of  Early  Warning  Network  (EWN)  systems  (i.e.  GBS  and 
LSDIS)  in  ADA  units.  The  risks  in  the  GBS  and  LSDIS 
acquisitions  may  or  may  not  be  directly  associated  with  the 
four  year  delay  witnessed  in  providing  initial  sensor  systems 
to  the  users .  It  provides  an  overview  of  the  two  sensors  and 
considers  the  risks  facing  the  GBS  and  LSDIS  systems.  It 
reviews  the  risk  management  approach  for  each  sensor  and 
presents  challenges  in  mitigating  NDI  risks  using 
developmental  item-based  policy,  practices,  and  risk 
management  tools  such  as  the  templates  found  in  DOD  4245. 7-M. 

Chapter  V  analyzes  DOD  NDI  risk  management  policy  and 
guidelines.  This  chapter  considers  the  challenges  and 
potential  solutions  for  effective  NDI  risk  management  based  on 
available  risk  management  tools.  The  tools  considered  are  the 
DOD  4245. 7-M  templates  and  a  NDI  risk  management  plan. 

Chapter  VI  makes  recommendations  and  proposes 
improvements  in  the  content,  value,  and  effectiveness  of  DOD 
NDI  risk  management  policies  and  guidelines  for  future 
programs.  These  emerge  from  the  application  of  lessons  in  the 


FAAD  Sensors  case  and  DOD  policy  review.  The  recommendations 
consist  of  revisions  to  risk  management  policies  and  practices 
in  NDI  acquisitions. 


II.   RISK  MANAGEMENT  OVERVIEW 

A.  PURPOSE 

This  chapter  introduces  risk  and  risk  management 
concepts.  This  risk  management  "primer"  uses  the  Program 
Management  Institute's  (PMI)  body  of  knowledge  on  risk 
management  as  the  source  of  this  basic  overview.  The  PMI 
approach  is  representative  of  the  general  body  of  knowledge 
concerning  risk  and  risk  management  that  may  be  found  in 
business  and  industry  today.  DSMC  risk  management  concepts 
are  subsequently  introduced.  The  chapter  then  presents  DOD 
and  U.S.  Army  risk  management  policy  which  acts  as  guidance 
for  PMs  as  they  structure  their  risk  management  programs. 

B.  THE  DEFINITION  OF  RISK 

What  is  risk?  Before  answering  this,  it  is  important  to 
understand  the  role  of  uncertainty.  Uncertainty  represents 
the  set  of  all  possible  outcomes  that  are  either  favorable 
(opportunities)  or  unfavorable  (risks)  .1  The  goal  of  risk 
management  is  to  move  uncertainties  in  an  acquisition  program 
away  from  the  risk  end  of  the  uncertainty  spectrum  and  towards 
the  opportunities  end. 

With  this  understanding  of  uncertainty,  risk  is  defined 
as  the  cumulative  probability  of  uncertain  occurrences 
adversely  affecting  a  program's  objectives.  Risk  itself 
consists  of  three  risk  factors.   These  are: 

1.  Risk  Events.  These  are  specific  occurrences 
that  can  hurt  progress  in  a  program.  The  consequences  of 
these  negative  events  are  described  predominantly  in  terms  of 
scope,  quality,  time,  and  cost  objectives  or  goals: 

a.   Scope  Risks  are  changes  in  scope  to  meet 


1  The  material  presented  in  this  chapter,  sections  B-F 
are  referenced  from  Wideman,  p. 1-2  to  p.B-1,  1992  except  as 
noted. 


program  objectives  (e.g.  regulatory  or  work  breakdown 
structure  changes) .  These  risks  include  both  ill -defined 
program  pla.n.3   and  requirements. 

b.  Quality  Risks  are  performance  or  technical 
failures.  These  risks  include  poor  quality  assurance  and  poor 
workmanship. 

c.  Schedule  Risks  are  failures  in  meeting  time 
estimates.  These  risks  consist  of  poor,  premature,  or 
optimistic  time  estimates,  scope  changes,  and/or  accelerated 
fielding  requirements . 

d.  Cost  Risks  are  failures  to  achieve  desired 
budget  targets.  These  risks  consist  of  poor,  premature,  or 
optimistic  cost  estimates  and/or  inadequate  cost  controls. 

The  type  of  acquisition  program  determines  the  type  of 
possible  risk  events  witnessed  and  their  influence  on  a 
program.  Most  risks  affect  more  than  one  program  objective 
with  the  predominance  of  risk  events  affecting  the  schedule 
and  cost  program  objectives. 

2.  Risk  Probability.   This  is  the  likelihood  of 
occurrence  of  each  of  the  risk  events . 

3.  Amount  at  Stake.   This  represents  the  severity 
of  the  consequences  if  a  particular  risk  event  occurs. 

C.   RISK  MANAGEMENT 

Risk  management  is  "the  art  and  science  of  identifying, 
assessing,  and  responding  to  project  risk  throughout  the  life 
of  a  project  and  in  the  best  interests  of  its  objectives" 
(Wideman,  p.  II-3,  1992).  Risk  management  is  a  continuous, 
integrated  process  conducted  throughout  the  life  of  an 
acquisition  program.  Its  goals  are  the  identification  of 
program  risk  and  the  development  of  risk  management  strategies 
to  minimize  or  avoid  those  risks.  Risk  management's  intent  is 
to  aid  proactive  decision-making  through  anticipatory  planning 
as  opposed  to  crisis  decisionmaking. 
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D.   IMPLEMENTING  RISK  MANAGEMENT 

A  program  office  implements  risk  management  to  assise 
planning  and  decisionmaking.  In  doing  so,  uncertainty,  risk, 
and  the  amount  at  stake  can  be  reduced  to  manageable  levels 
throughout  the  program  life. 

Risk  management  in  business  stresses  four  phases: 
identification,  assessment,  response,  and  documentation  (see 
Figure  1  below) . 
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Figure  1  The  Four  Phases  of  Risk  Management   (Source 
Wideman,  Chapter  III,  1992) 
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1.   Program  Risk  Identification 

Risk  identification  is  the  identification  of  specific 
characteristics  thought  to  contribute  to  uncertainty  in 
program  objectives  (Defense  Advanced  Research  Projects  Agency, 
p.  6,  1983) .  This  first  phase  seeks  to  identify  all  the 
possible  risks  which  may  significantly  impact  a  program's 
success.  Examples  of  sources  of  risk  are:  unanticipated 
Government  intervention,  unpredicted  changes  or  shifts  in 
economic  markets,  inappropriate  program  structure,  scheduling 
delays,  unanticipated  resource  problems,  changes  in 
technology,  changes  in  performance  requirements  or  design, 
increased  technical  complexity,  and  contractual  difficulties 
evolving  from  the  type  of  contract  or  data  rights  issues. 

Risk  identification  requires  PMs  to  be  familiar  with  the 
program's  scope,  product,  and  objectives.  Risks  are  matched 
to  the  program  objectives  they  affect  and  then  screened  by 
their  type  and  source.  Identification  should  be  completed 
early  in  a  program's  life  to  facilitate  informed 
decisionmaking  and  provide  risk  early  warning  signals.  PMs 
can  identify  sources  of  risk  and  understand  them  better  using 
a  multi-functional  team  to  examine  program  details  and 
determine  their  risks  and  uncertainties.  These  can  then 
provide  PMs  initial  risk  estimates. 


2.   Program  Risk  Assessment 

Risk  assessment  is  the  "examination  of  program  elements, 
using  experts,  in  order  to  synthesize  a  model... (it)  is  a 
means  of  reducing  the  knowledge  of  program  elements  to 
quantitative  expressions  of  uncertainty"  (DSMC,  p.  II-2, 
1983)  .  This  phase  seeks  to  analyze  the  individual  elements 
that   create   uncertainty   or   problems,   determine   the 
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relationships  among  those  elements,  and  then  formulate  a  model 
to  study  the  problem  (Ingalls,  p. 28,  1983)  .  Quantification 
then  follows  to  develop  mathematical  descriptions  of  those 
relationships  posed  in  the  model. 

Risk  assessment  consists  of  two  primary  steps:  assessing 
and  assigning  probabilities  to  risk  events  and  analyzing  the 
consequences  on  the  project.  Assessment  includes  assigning 
probabilities  to  undesired  or  unfavorable  events  (risks)  and 
examining  their  affect  on  program  objectives.  This  again  can 
effectively  be  done  using  a  multifunctional  team.  A  risk 
assessment  team  rates  program  risks  by  type,  impact,  and 
probability  of  occurrence  using  subjective  or  quantitative 
estimates.  Risk  ratings  normally  consist  of  low,  medium,  and 
high  based  on  their  severity  of  impact.  Risk  assessment 
treats  risks  as  probabilities  of  unfavorable  events  that  occur 
within  the  range  of  total  success  and  total  failure.  This 
range  establishes  a  risk  baseline  and  the  ability  to  benchmark 
a  program's  risk  status  against  program  objectives  over  a 
program's  life. 

Quantitative  risk  assessment  is  called  risk  analysis.  It 
models  and  quantifies  risk  events  and  their  impact  on  program 
objectives.  Analysis  concentrates  primarily  on  medium-  and 
high- rated  program  risks  with  the  highest  likelihood  of 
occurrence.  Contingency  allowances,  such  as  management 
reserves,  cover  risks  with  lower  severity.  Together,  risk 
assessment  and  the  risk  analysis  which  supports  it  provide  a 
risk  profile  or  picture  that  the  PM  can  use  to  make  informed 
program  decisions. 

Risk  analysis  requires  the  PM  to  estimate  the  degree  of 
uncertainty  expected  early  in  a  program.  This  may  be  done 
using  probability  distributions,  probability  tree  diagrams, 
modeling,  or  sensitivity  analysis  (as  opposed  to  simply  using 
subjective  assessment  or  "gut  feeling").  Risk  analysis 
modeling   techniques   that   support   this   phase   include 
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probability  distribution  analysis,  the  Delphi  Method,  Monte 
Carlo  simulations,  and  decision  theory. 

Risk  analysis  should  accompany  risk  assessment  early  in 
a  program's  life  cycle  and  can  be  tailored  to  the  program.  It 
is  an  iterative,  continuous  cycle  used  to  establish  or  update 
the  risk  baseline.  Its  value  is  as  an  integral  part  of 
program  management  planning. 

3 .   Risk  Response 

Risk  response  is  program  manager  actions  and  strategy 
initiated  once  risks  have  been  identified  and  assessed.  This 
phase  establishes  strategies  to  effectively  handle  risk. 
Responses  include: 

1.  Acceptance.  Management  takes  no  action  after 
recognition  (management  responds  by  adjusting 
operations  or  modifying  objectives) . 


2.  Avoidance.   Management  takes  proactive  steps,  makes 
contingency  plans,  or  selects  lower- risk  alternatives. 


3.  Reduction/Minimization.  Management  uses  alternative 
approaches  such  as  workarounds  or  collection  of  data 
and  information  to  facilitate  contingency  planning. 


4.  Sharing.    Management   shares   risk  through  joint 
agreements  or  contract  agreements. 


5.  Transfer.   Management  deflects  risk  via  a  contract 
clause,  liability  or  loss  insurance,  or  warranties. 


6.  Absorption.     Management   relies   on   contingency 
allowances  or  reserves. 
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7.  A  combination  of  the  above  methods. 

Program  managers  can  hold  periodic  reviews  to  monitor 
changes  in  program  risks  and  update  proposed  responses . 


4.   Risk  Data  Collection,  Application,  and 
Documentation 


This  final  risk  management  phase  builds  a  reliable 
database  to  allow  continued  risk  evaluations  for  current 
programs  and  as  reference  sources  for  future  programs.  Risk 
management  data  sources  are  established  through  historical 
databases,  current  databases,  and  post -program  reviews  and 
archives.  These  aid  the  estimation,  comparison,  updating,  and 
information  exchange  needs  of  the  program  manager.  Data- 
building  facilitates  and  simplifies  risk  and  program  decisions 
particularly  when  multifunctional  teaming  and  high  turnover 
prevail  in  a  program  office. 

Risk  management  is  a  continuous,  iterative  process 
conducted  throughout  the  lifecycle  of  a  system's  acquisition. 
Managing  the  uncertainties  of  a  program  and  the  consequences 
of  unfavorable  events  is  a  difficult  task.  DOD  faces  its  own 
acquisition  risks.  Its  risk  management  approach  is  the  focus 
of  the  next  section. 

E.   RISK  MANAGEMENT  IN  THE  DEPARTMENT  OF  DEFENSE 

1.   Risk  Definition 

DOD  defines  risk  somewhat  differently  from  the  definition 
presented  earlier.  The  DOD  definition  considers  subjectivity 
and  impacts  made  on  program  objectives  rather  than  simply 
considering  its  impact  on  the  whole  program.  It  defines  risk 
as  "a  subjective  assessment  made  regarding  the  likelihood  or 
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probability  of  not  achieving  a  specific  objective  by  the  time 
established  with  the  resources  provided  or  requested"  (DODI 
5000.2,  p.  15-15,  1991) .  Risk  is  characterized  in  the 
following  ways:2 

a.  Technical  Risk.  This  is  performance -based  risk 
in  the  development  of  a  new  design  for  better  performance  or 
with  new  constraints  on  existing  performance.  It  includes 
operational  capability,  reliability,  and  producibility. 
Examples  relating  to  technical  risk  are  development  and  design 
based  on  unproven  technologies,  requirements  changes,  and/or 
configuration  changes. 

b.  Supportability  Risk.  This  is  performance -based 
risk  in  fielding  and  providing  maintenance  support  to  new 
systems.  It  results  from  technical  or  programmatic  risks. 
Examples  include  unforeseen  training  requirements,  and/or 
failure  to  achieve  reliability,  availability,  and 
maintainability  requirements. 

(Technical  and  Supportability  Risk  appear  to  correspond 
to  Performance  Risk  defined  in  Section  B) . 

c.  Programmatic  Risk.  This  is  environment -based 
risk  from  the  impact  of  events  beyond  a  PM' s  control .  These 
include  higher  level  decisions  and/or  turbulence  introduced 
from  outside  the  program  (e.g.  Congress)  and  resource  changes 
that  can  influence  program  success.  Additional  examples  are 
material  and  personnel  shortages  or  turbulence,  contractor 
instability,  and/or  funding  changes. 

(Programmatic  Risk  appears  to  correspond  to  Scope  Risk 
defined  in  Section  B) . 

d.  Cost  Risk.  This  is  the  risk  of  cost  projections 
not  being  realistic  or  of  the  program  not  progressing 
efficiently  to  meet  established  cost  objectives.   This  often 


2  The  material  in  sections  E-H  is  referenced  from  DSMC's 
Risk  Management  Concepts  and  Guidance,  p. 3 -4  to  p.C-7,  1989 
except  where  noted. 
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results  from  sensitivity  to  technical,  supportability,  or 
programmatic  risks.  Examples  are  increased  overhead  rates  and 
cost  estimation  errors. 

e.  Schedule  Risk.  This  is  the  risk  of  schedule 
estimates  being  too  optimistic  or  of  the  program  not 
developing  effectively  to  achieve  established  baselines.  This 
often  results  from  sensitivity  to  technical,  cost, 
supportability,  or  programmatic  risks.  Examples  include 
problems  resulting  from  concurrent  development  and  production 
and  schedule  estimation  errors. 

According  to  the  DSMC,  technical,  supportability,  and 
programmatic  risks  act  as  sources  of  risk  while  cost  and 
schedule  risks  are  indicators  of  those  sources  of  unfavorable 
events.  These  five  types  of  risk  act  interdependently  to 
affect  a  program's  performance. 


2.   Risk  Management 

DOD  defines  risk  management  as  "a  systematic  approach  to 
identifying,  analyzing,  and  controlling  areas  or  events  with 
a  potential  for  causing  unwanted  change.  It  is  an  integral 
part  of  the  overall  program  management  effort"  (PM  Notebook, 
p. 4. 5-3,  1992).  This  definition  differs  from  the  business 
definition  in  that  it  stresses  a  systematic  approach  rather 
than  "art  and  science".  DODI  5000.2  adds  that  these  actions 
are  taken  to  bring  risk  "to  an  acceptable  level  in  selected 
areas  (e.g.  cost,  schedule,  technical,  producibility,  etc.) 
and  the  total  program  (DODI  5000.2,  p.  15-15,  1989).  Risk 
management  identifies  and  evaluates  the  vulnerable  areas  of  a 
program.  Its  purpose  is  to  provide  a  means  of  comparing  risk 
management  performance  to  a  standard  and  tracking  risk- related 
information. 


17 


3 .   The  Defense  Risk  Management  Process 

A  PM' s  risk  management  process  sets  objectives  and 
repeatedly  assesses  the  program  for  obstacles  preventing  those 
objectives  from  being  attained.  The  DOD  risk  management 
process  is  similar  to  the  basic  process  described  earlier.  It 
requires  that  criteria  or  a  set  of  standards  with  which  to 
judge  successful  risk  assessment  and  mitigation  be  established 
early  in  the  program's  life.  The  DOD  process  consists  of  a 
different  set  of  phases  from  the  basic  process.  It  consists 
of  planning,  assessing,  analyzing,  and  handling  risk  (see 
Figure  2)  .  This  structure,  like  the  basic  approach,  considers 
both  internal  and  external  circumstances  affecting  a  program's 
objectives.  It  likewise  allows  the  PM  to  formulate 
alternative  courses  of  action  and  to  make  rational  decisions 
on  monitoring  and  controlling  the  outcomes  of  program  events. 

The  four  defense  risk  management  process  components  are: 

a.  Risk  Planning.  This  first  phase  defines  the 
program's  present  and  future  risks.  This  establishes  the 
need,  purpose,  and  requirements* for  managing  particular  risks 
within  a  program  to  eliminate,  minimize,  or  contain  them.  Risk 
identification  in  this  phase  gives  a  "plain  English" 
description  of  risk  events  that  can  prevent  program  goals  from 
being  achieved.  The  program's  Work  Breakdown  Structure  is  a 
favorite  tool  to  provide  a  risk  identification,  rating,  and 
analysis  structure.  Risk  rating  schemes  include  lessons 
learned,  expert  interviews,  baselines,  templates,  analogy 
comparisons,  program  document  evaluations,  technology 
evaluations,  and  contractor  risk  evaluations.  This  phase 
corresponds  with  the  business  Risk  Identification  phase. 

b.  Risk  Assessment.  This  is  the  examination  of  a 
program  objective  and  assessment  of  those  risk  events  posing 
potential  problems.  Risk  ratings  and  a  risk  baseline  similar 
to  those  described  earlier  give  PMs  a  means  of  datermining  a 


18 


Figure  2  The  DOD  Risk  Management  Process  (Source:  DSMC  Risk 
Management  Concepts  and  Guidance,  Chapter  5,  19  89) 


program's  risk  status.  The  combination  of  assessment  and 
analysis  shows  points  in  a  program  where  the  consequences  of 
risk  are  likely  to  occur,  their  severity,  and  what  drives 
them.  The  DOD  4245. 7-M  risk  templates  are  common  DOD  tools 
for  assessing  program  risk.  Risk  quantification  begins  in 
this  phase.  Whereas  business  groups  risk  assessment  and  risk 
analysis  together,  the  defense  risk  management  process 
separates  risk  assessment  and  risk  analysis. 

c.  Risk  Analysis .  Risk  analysis  is  an  examination 
of  the  changes  in  consequences  caused  by  changes  to  risk  input 
variables.   Analytical  and  quantitative  methods  are  used  to 


19 


demonstrate  the  potential  impacts  of  risk  consequences  on  a 
program.  These  methods  include  simulations,  network  models 
(PERT,  CPM) ,  cumulative  probability  distributions,  life  cycle 
cost  programs,  and  quick  reaction  modeling.  These  tools  allow 
a  PM  to  develop  risk-handling  options  and  techniques. 

Risk  analysis  provides  valuable  outputs.  One  example  of 
these  is  the  watchlist  (PM  Notebook,  p. 4. 5-4,  1992).  A 
watchlist  is  a  signalling  mechanism  for  risk-based  events  or 
outcomes.  The  list  consists  of  areas  of  risk,  their  expected 
impacts,  and  ways  to  detect  and  manage  each  prioritized  risk 
area.  It  acts  as  a  worksheet  to  track  risk  management 
progress  and  to  show  where  the  PM  can  make  acceptable  risk- 
program  performance  tradeoffs. 

d.  Risk-Handling.  Handling  risk  areas 
completes  the  DOD  version  of  the  risk  management  process. 
This  is  the  formulation  of  management  options  to  control  or 
minimize  areas  of  risk.  DOD  risk-handling  measures  are 
categorized  as  avoidance,  control  and  prevention,  assumption 
of  consequences,  transfer  of  responsibility,  and  knowledge  and 
research.  Risk  control  processes  are  the  most  common  risk- 
handling  measures.  They  consist  of  event-based,  Government 
reviews  and  audits  paralleled  by  contractor  systematic 
analyses,  reviews,  and  tests.  Such  continuous  monitoring  and 
corrective  actions  must  be  coordinated  and  integrated  into 
program  plans  and  documents .  This  phase  corresponds  to  the 
business  phases  of  Risk  Response  and  Risk  Documentation. 

The  DOD  risk  management  process  is  sequential  and 
continuous  much  like  the  risk  management  process  described 
earlier.  Risk  management  evaluates  what  is  achievable  against 
what  is  planned;  identifies  risk  areas  using  lessons  learned, 
risk  templates,  work  breakdown  structures,  and  expert 
interviews;  rates  and  quantifies  risks;  analyzes  their 
severity  of  impact  on  program  completion;  and  offers  risk- 
handling  options  and  tools.   Once  risk  handling  options  are 
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implemented,  the  PM  should  document  risk  areas  for  future 
programs'  benefit.  A  means  of  documenting  the  PM's  risk 
management  program  is  through  the  publication  of  a  risk 
management  plan. 

F.   DOD  RISK  MANAGEMENT  POLICIES 

Risk  management  policies  for  acquisition  programs  are 
described  in  the  DOD  5000 -series  policies.  The  purpose  of  the 
5000 -series  is  to  provide  acquisition  program  policy, 
regulations,  instructions,  and  format  for  PMs  -  to  include 
their  risk  management  programs.  DOD  Directive  (DODD)  5000.1 
is  the  first  authority  for  risk  management  policy.  According 
to  DODD  5000.1: 

Program  risks  and  risk  management  plans  shall  be 
explicitly  assessed  at  each  milestone  decision 
point  prior  to  granting  approval  to  the  next 
acquisition  phase.  This  includes:  early 
identification  of  critical  parameters,  technology 
demonstrations  to  reduce  risk,  test  and  evaluation 
to  determine  system  maturity  and  identify  risks, 
contractor  risk  plans  and  trade-offs  to  keep  risk 
at  acceptable  levels. 

The  Directive  points  out  that  risk  assessment  and 
reduction  plans  are  required  ^as  part  of  the  acquisition 
process  to  ensure  sound  program  management  decisions  (DODD 
5000.1,  p.  1-5,  1991) . 

DODI  5000.2  follows  DODD  5000.1  with  more  detailed 
guidance  on  managing  risk.  DODI  5000.2  Part  5B,  published  1 
February  1991  states: 

A  risk  management  program  shall  be  established  for 
each  acquisition  program  to  identify  and  control 
performance,  cost,  and  schedule  risks  using  the 
areas  of  risk  identified  in  DOD  4245. 7-M.  The  risk 
management  program  must  include  provisions  for 
eliminating  these  risks  or  reducing  them  to 
acceptable  levels.  Industry  participation  in  risk 
management  is  essential. 
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DOD  4245. 7-M  is  a  document  containing  risk  management 
templates.  These  templates  are  tools  for  assessing  and 
reducing  program  risks. 

DODI  5000.2  elaborates  on  risk  management  in  Part  5B, 
Change  1,  dated  26  February  1993.  It  states  "risks,  risk 
reduction  measures,  rationale,  and  assumptions  made  in 
assigning  risk  ratings,  and  alternative  acquisition  strategies 
will  be  explicitly  assessed  at  each  milestone  decision  point." 

Finally,  the  5000. 2 -M  reference  for  acquisition 
documentation  and  formats  addresses  the  risk  management  area. 
DOD  5000. 2 -M  indicates  "The  Integrated  Program  Summary  (IPS) 
for  each  milestone,  shall  assess  the  risks  with  respect  to  the 
threat,  technology,  design,  and  engineering  support, 
manufacturing,  cost  and  schedule.  The  IPS  shall  summarize  the 
actions  being  taken  to  mitigate  those  risks." 

G.   U.S.  ARMY  RISK  MANAGEMENT  POLICIES 

U.S.  Army  leadership  in  and  treatment  of  risk  management 
in  research,  development,  and  acquisition  programs  began  in 
1970  (Muldrow,  p.  28,  1985)  .  In  DA  PAM  11-2,  Research  and 
Development  Cost  Guide  for  Army  Material  Systems  (May  1976) , 
paragraph  3.5,  PMs  are  encouraged  to  manage  risks  by  treating 
costs  as  ranges  rather  than  point  estimates.  The  Army 
Development  Acquisition  and  Readiness  Command  (now  known  as 
the  Army  Material  Command)  Regulation,  DARCOM-R  11- 1  (April 
1976)  on  systems  analysis,  focuses  extensively  on  risk 
analysis  guidelines.  Appendix  D  of  this  DARCOM-R  prescribes 
its  Decision  Risk  Analysis  Guidelines  for  PMs  as  follows: 

1.  a.  Define  the  problem 

b.  Establish  the  decisionmakers'  preferences  for 
trade-offs  between  cost,  schedule,  and/or  performance 

2.  Establish  the  alternatives 

3 .  Define  the  events 

4.  Collect  the  data 
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5 .  Determine  the  program  risks 

6.  Select  the  best  alternatives 

7.  Perform  sensitivity  analysis 

8.  Communicate  the  results 

These  steps  are  consistent  with  guidelines  suggested  in 
both  the  DSMC's  risk  management  publication  and  the  PMI  body 
of  knowledge. 

H.   SUMMARY 

Through  the  DODD  5000.1  and  DODI  5000.2,  the  Defense 
Department  provides  risk  management  material  and  guidance. 
The  5000 -series  was  intended  to  communicate  risk  management 
policy  and  guidelines  for  all  PMs . 

The  problem  with  DOD  risk  management  policy  is  that 
policy  and  guidelines  written  for  the  Cold  War  defense 
environment  was  oriented  toward  full-scale,  developmental 
programs  and  the  five -phase  acquisition  cycle.  NDI  are  the 
exception  to  these  rules.  Their  risk  management  programs  must 
be  tailored  because  no  explicit  NDI  risk  management  exists. 
The  new  dimension  of  risk  management  of  NDI  acquisitions,  the 
DOD  policies  concerned,  and  the  difficulties  involved  are  the 
focus  of  the  next  chapter. 
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III.   RISK  MANAGEMENT  IN  NDI  ACQUISITIONS 

A.  PURPOSE  AND  BACKGROUND 

This  chapter  examines  NDI  acquisitions  and  the  conditions 
that  make  NDI  acquisition  strategies  attractive.  It  defines 
what  NDIs  are  and  are  not,  their  advantages  and  disadvantages, 
and  the  acquisition  policies  governing  them.  This  sets  the 
stage  to  study  how  DOD  risk  management  policies  and  practices 
address  NDIs  and  support  the  PM  managing  a  NDI  acquisition 
program. 

Commercial  business  and  DOD  have  new  roles  and  face  new 
trends  compared  to  those  of  the  Cold  War  status  quo.  During 
the  Cold  War,  the  defense  establishment  drove  the  pace  of 
technological  innovation.  Several  trends  have  since  shaped 
new  roles  (Borrus,  p. 41,  1990) .  First,  civilian  technology  no 
longer  trails  military  technology  as  it  did  for  several 
decades  during  the  Cold  War  era.  Next,  commercial  production 
features  greater  quality  and  efficiencies  in  flexible 
manufacturing,  multiple  product  lines,  just- in- time 
inventories,  and  lower  volume  production  runs.  Finally,  many 
critical  defense  technologies  have  considerable  overlap  with 
commercial  technologies.3  The  need  to  modernize  and  retain 
state-of-the-art  capabilities  for  military  forces  in  an  era  of 
capped  military  spending  supports  the  purchase  of  commercial 
products  for  military  use.  The  primary  means  of  acquiring  and 
fielding  these  products  is  with  NDIs. 

B.  THE  DEFINITION  OF  NDI 

The  basic  definition  of  NDI  is  "a  broad  term  covering 
material  available  from  a  variety  of  sources,  with  little  or 
no  development  effort  required  by  the  government"  (DSMC  NDI 


3   The  material  in  this  chapter  references  the  DSMC  NDI 
Acquisition  Guide,  p.xiii  to  p. 78,  1992  except  as  indicated. 
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Acquisition  Guide,  p.  3,  1992).  NDIs  are  an  option  for 
fielding  systems  more  quickly  and  cheaply  than  with  a  new, 
developmental  item.  NDI  acquisition  strategies  look  to 
commercial  vendors  and  federal,  state,  local,  and  foreign 
governments  as  sources  for  the  desired  item.  They  also 
consider  any  commercial  item  in  design,  development,  and/or 
production  but  not  yet  available  for  commercial  use  (Prueitt, 
1994)  . 

The  U.S.  Congress  defines  a  NDI  as: 

1.  Any  item  available  in  the  commercial  marketplace. 


2.  Any  previously  developed  item  in  use  by  the  U.S 
Government  or  cooperating  foreign  governments. 


3  .  Any  item  of  supply  needing  only  minor  modifications  to 
meet  DOD  requirements. 

The  military  services  define  NDIs  more  specifically.  The 
Army  version  distinguishes  NDI  in  three  categories  (Quindlen, 
p.  53,  1989) : 

1.  Category  A  (Basic  NDI)-:  Off-the-shelf  items  to  be 
used  in  the  same  environment  for  which  items  were 
designed  with  little  or  no  development  or  modification 
required. 


Category  B  (NDI  Adaptation) :  Off-the-shelf  items  to  be 
used  in  an  environment  different  than  that  for  which 
the  items  were  designed  with  some  development 
required.  These  include  products  requiring 
hardening,  strengthening  and/or  related  modifications . 
This  category  is  sometimes  referred  to  as  "Ruggedized 
NDI"  . 


3.  Category  C  (NDI  Integration) :  Integration  of  existing 
componentry  and  the  essential  engineering  effort  to 
accomplish  systems  integration  with  research  and 
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development  to  integrate  systems .   This  category  is 
sometimes  labelled  "Militarized  NDI". 

Confusion  sometimes  arises  in  distinguishing  a  NDI  from 
a  commercial-off- the- shelf  (COTS)  item.  A  COTS  item  makes  up 
only  a  portion  of  DOD- classified  NDIs .  COTS  items  are 
commercial  hardware/software  items  not  modified  by  the 
government,  items  that  are  in  the  commercial  inventory  or 
production,  that  have  proved  their  performance  in  a  similar 
environment,  that  have  an  existing  support  structure,  have  an 
internal  support  structure,  have  an  internal  configuration 
that  flows  with  commercial  changes  and  generally  are 
integrated  with  other  hardware/software  items  to  become  part 
of  a  system  or  subsystem  capability. 

C.   USING  NDI  FOR  REDUCING  PROGRAM  RISKS 

NDIs  have  become  increasingly  important  in  equipping  our 
services.  NDI  acquisitions  are  more  prevalent  as  an 
alternative  to  costly,  lengthy,  and  more  risky  developmental 
programs.  The  lower  risks  commonly  associated  with  cost, 
schedule,  and  technical  performance  make  NDI  acquisitions 
politically  and  programmatically  popular. 

NDIs  are  considered  a  means  of  mitigating  program  risks. 
Risk  mitigation  involves  the  assessment  and  response 
activities  of  risk  management  which  were  outlined  in  the 
previous  chapter.  Risk  mitigation  is  implemented  through  the 
shortened  or  eliminated  Demonstration  and  Validation  (DEMVAL) 
Phase  and  Engineering  and  Manufacturing  Development  (EMD) 
Phase  of  the  NDI  acquisition  cycle.  The  streamlined  NDI 
acquisition  cycle  reduces  cost  and  schedule  risks.  However, 
NDI  acquisitions  do  not  eliminate  risk  but  may  in  fact 
introduce  new,  significant  risks.  These  risks  will  be 
examined  shortly. 

An  effective  NDI  acquisition  program  must  fulfill  the 
Service's  needs  by  fielding  mature  technology  that  preserves 
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combat  readiness  while  satisfying  popular  expectations.  The 
three  features  which  attract  substantial  DOD  and  Congressional 
popular  support  to  NDI  alternatives  but  which  challenge  the  PM 
are: 

1.  Meeting  user  requirements  based  on  available 
commercial  market  solutions. 

2.  Tailoring  user  requirements  to  suit  NDI  acquisition 
strategies  while  avoiding  "goldplating" . 

3.  Opening  up  greater  access  to  state-of-the-art 
technology  for  DOD  while  keeping  pace  with  changing  threats, 
emerging  technologies,  and  innovative  combat  systems. 

To  DOD  and  Congressional  decisionmakers,  NDIs  appear  to 
be  a  low- risk  approach  for  meeting  the  Services'  needs  and 
requirements.  The  danger,  however,  is  in  forcing  NDI 
solutions  on  PMs  and  their  customers  (the  user  community)  , 
when  those  NDI  neither  meet  requirements  nor  transition 
effectively  from  commercial  to  battlefield  environments. 

D.   NDI  BENEFITS  AND  POTENTIAL  RISKS 

1.   Benefits 

NDI  acquisitions  have  many  benefits  which  make  the 
transition  of  commercial  technology  to  military  use  a  popular 
course  of  action.  A  NDI  acquisition  provides  an  existing  or 
modified  system  to  meet  a  stated  requirement  in  a  cost- 
effective  and  timely  manner.   The  benefits  NDIs  offer  are: 


Time 


Decreased  acquisition  cycles,  shorter  procurement 
lead  times,  and  reduced  testing  requirements  often  result  in 
quicker  system  delivery  and  fielding  (Cain,  p.  22,  1993) .  See 
Figure  3 . 
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Cost 


Minimized  research  and  development  costs  reduce 
expected  lifecycle  costs.  These  costs  savings  stem  from  using 
commercial  specifications  and  the  competitive  pricing  inherent 
in  commercial  markets  (Cain,  p.  23,  1993) 

c.  Technical   Performance 

State-of-the-art  technology  from  the  commercial 
marketplace  meets  operational  requirements  with  limited  design 
and  development.  NDI  programs  introduce  the  acquisition  of 
mature  technologies,  with  validated  and  established  production 
techniques,  and  high  quality  standards.  Reliability, 
availability,  and  maintainability  (RAM) ,  and  supportability 
data  are  already  established  as  part  of  commercial  testing  and 
market  demands  (Garcia,  p.  8,  1991). 

d.  Risk  Reduction 

Risks  to  development  costs,  program  duration,  and 
leading  edge  technical  performance  are  decreased.  NDI  limits 
risks  with  an  established  cost  structure,  minimized  production 
start-up  costs  with  currently  operating  production  lines, 
economies  of  scale  where  the  Government  is  not  the  sole 
customer,  and  the  competitive  pressures  of  commercial  markets 
on  reasonably-priced,  high  volume  production  (Vandeviere,  p. 
13,  1994)  . 

e .  Con  tra  c  ting 

Simplified  contracts  often  result  from  procurement 
of  established  products.  The  increased  use  of  Fixed-Price- 
type   contracts   puts   more   risk   responsibility   on   the 
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Figure  3  Notional  Developmental  and  Non- developmental  Item 
Timelines  (Source:  Vandeviere,  Chapter  2,  1994) 


contractors'  shoulders  while  reducing  Government  risk  (GAO,  p. 
11,  1989)  . 

2.   Potential  Risks 

A  developmental  item  PM  traditionally  faces  cost, 
schedule,  performance,  support  ability,  and  programmatic  risks 
whereas  the  NDI  PM' s  program  risks  are  more  unique .  NDIs 
typically  reduce  cost  and  schedule  risks.  The  risks  described 
below  affect  the  technical  performance,  supportability,  and 
programmatic  objectives  in  a  NDI  program.  In  reviewing  the 
literature,  the  author  has  determined  that  the  following  five 
areas  are  the  most  pressing  potential  risks  facing  NDI 
acquisition  programs:  Requirements,  NDI  Acquisition  Management 
Environment,  Specifications  and  Standards,  Test  and 
Evaluation,  and  Integrated  Logistics  Support.  If  not  properly 
addressed,  these  risks  may  eas'ily  affect  cost  and  schedule 
ob j  ectives  as  well . 
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a.  Requirements 

"Initial  (Operational)  Requirements  Documents  (ORD) 
are  developed  without  careful  consideration  of  what  is 
currently  available  as  NDI"  (DSMC  NDI  Acquisition  Guide,  p. 
14,  1992).  Requirements  risks . surf ace  when  the  operational 
community  develops  an  ORD  focused  entirely  on  operational 
need,  independent  of  market  solutions  available  to  meet  those 
needs .  NDI  requirements  risk  can  arise  from  users  developing 
an  ORD  based  on  available  commercial  technology  rather  than 
available  commercial  products.  The  user  proceeds  to  mix  and 
match  the  desired  functions  or  capabilities  of  several 
products  into  one  ORD  when  no  existing  item  can  meet  user 
requirements  without  making  extensive  modifications.  On  the 
other  hand,  the  user  may  accept  an  available  NDI  solution 
which  does  not  strictly  comply  with  all  the  specified 
requirements . 

Requirements  risk  affects  all  of  a  program's 
objectives.  This  risk  in  NDI  acquisitions  often  indicates  a 
lack  of  flexibility  in  ORD  development.  The  risk  grows  when 
the  requirements  generation  process  does  not  include  the  PM. 
The  PM' s  responsibility,  as  the  material  developer,  is  to 
respond  to  user  needs  but  to  be  fully  conscious  of  technical 
risks,  af f ordability  constraints,  and  schedule  impacts.  The 
user's  responsibility  is  to  be  realistic  in  stated 
requirements,  weigh  proven  capability  and  rapid  deployment 
benefits  against  performance  limitations,  and  negotiate 
acceptable  trade-offs  with  the  PM.  Too  often  these 
responsibilities  are  not  integrated  or  communicated. 
Requirements  risk  affects  a  user's  desire  or  ability  to  back 
off  of  validated  ORD  requirements.  When  minor  modifications 
are  pursued  to  comply  with  stated  requirements,  the  NDI  may 
take  on  extensive  and  costly  developmental  item 
characteristics  and  defeat  the  purpose  of  the  NDI  acquisition 
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strategy  (Cain,  p. 24,  1993)  . 

Insufficient  or  ineffective  market  analysis 
accentuates  the  problem.  Market  analysis  may  not  be  detailed 
enough  to  identify  technologies  and  potential  contractors. 
The  user/PM  team  then  cannot  evaluate  the  strengths  of  a  NDI 
acquisition  strategy  compared  to  a  developmental  one.  Market 
analysis,  when  properly  used,  contains  both  broadly- scoped 
market  surveillance  to  canvas  technologies  and  product 
developments  and  the  narrower- focused  market  investigation 
used  to  determine  what  technology  or  products  meet  user  needs. 
Insufficient  commitment  of  effort  and  resources  to  teaming  and 
market  analysis  leads  the  user  and  PM  to  either  neglect 
potential  alternatives  or  falsely  conclude  that  no  commercial 
alternative  exists  to  fulfill  user  requirements. 

Jb.  NDI  Acquisition  Management  Environment 

The  NDI  acquisition  management  environment  confronts 
the  PM  with  the  complexities  of  multiple  stakeholders  (DOD, 
Congress,  the  Service,  and  the  user) ,  internal  and  external 
perceptions  and  attitudes  (cultural  resistance)  towards  NDIs, 
commercial  standards  and  practices,  developmental ly- based 
paperwork  requirements,  audits,  and  inspections,  and  mistrust 
among  the  NDI  acquisition  participants.  These  elements  create 
programmatic  risks  for  the  PM. 

The  successful  achievement  of  cost,  schedule, 
performance,  and  supportability  "objectives  depends  on  the  PM's 
interaction  and  effective  teaming  with  the  prime  contractor  as 
well  as  with  the  user  community.  Trade-offs  between  ORD- based 
requirements  and  cost  and  schedule  constraints  on  the  program 
may  become  sources  of  conflict.  The  Government's  desire  to 
buy  as  opposed  to  the  contractor's  unwillingness  to  provide 
adequate  technical  data  may  become  a  major  source  of  friction. 
The  contractor's  unwillingness  to  allow  quality  control 
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oversight  and  qualification  of  its  manufacturing  and 
production  processes  may  contribute  to  a  contentious 
relationship  with  the  PM  and  the  Government.  NDI  performance 
expectations  may  not  be  realistic  to  both  parties.  Lack  of 
communication  before  and  after  program  reviews  may  prevent 
clearly- defined  agreements  and  changes  from  being  implemented 
during  the  NDI  acquisition  cycle.  With  the  shortened  NDI 
acquisition  cycles,  the  communication  and  feedback  loops  may 
not  develop  and  mature.  The  results  may  become  mistrust  and 
inappropriate  trade-off  decisions. 

NDI  acquisition  success  requires  changing 
traditional,  sometimes  antagonistic  procurement  mindsets  in 
the  acquisition  and  user  communities.  The  NDI  environment  may 
not  be  conducive  to  effective  communication,  coordination,  and 
cooperation.  The  willingness  to  objectively  assess  and  choose 
between  the  best  technology  available  and  a  full-scale 
development  start-up  requires  open-minded  acquisition  team 
members.  The  team  members'  responsiveness  to  new  acquisition 
strategies  requires  overcoming  ingrained  developmental  program 
biases.  Such  an  atmosphere  may  require  extensive  training  and 
education  (Garcia,  p.  26,  1991) . 

Political   and  defense   interests  may  drive  an 
acquisition  towards  a  NDI  acquisition  strategy  when  it  is  not 
appropriate.   This  confines  the  PM' s  efforts  to  the  "NDI  box 
(with)  NDI  boundaries  and  NDI  criteria"  when  that  NDI  option 
may  not  meet  the  user's  requirements.   The  appeal  of  NDIs  may 
foster  "zero  defects"  expectations  whereby  few,   if  any, 
program  or  contractual  problems  will  be  accepted,  especially 
if  the  user  has  experienced  previous  acquisition  program 
failures.  Given  constrained  budget  and  resource  environments, 
programming  and  budgeting  decisions  from  outside  the  PM' s 
sphere  of  influence  may  create  programmatic  risks  themselves. 
Lastly,  NDI  streamlining  may  ignore  useful  development 
program  management  tools  such  as  program  reviews  and  design 
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reviews.  These  tools  may  be  ignored  to  meet  accelerated 
schedules  and  to  minimize  or  streamline  the  commitment  of 
resources,  funds,  and  personnel. 

c.  Specifications  and  Standards 

Specifications  and  standards  serve  as  a  means  of 
requirements   verification   in   a   product's   design   and 
development  but  they  contribute  to  technical  performance  risks 
in  a  program.   The  risk  in  specifications  and  standards  (S&S) 
concerns  the  use  of  military  as  opposed  to  commercial  S&S. 

Military  specifications  (MILSPECS)  and  military 
standards  (MILSTDS)  draw  criticism  from  industry  as  being 
excessively  demanding,  outdated,  incompatible  with  NDIs,  and 
unnecessary  in  light  of  the  detail  presently  existing  in 
commercial  item  descriptions  (CIDs)  .  In  some  cases  such  as 
nuclear  delivery  systems,  strict  MILSPECS  and  MILSTDS  are 
justifiable,  but  the  tendency  is  to  go  beyond  performance 
(form,  fit,  and  function)  requirements  to  describe  the  design 
and  manufacturing  processes  themselves.  The  S&S  risks 
materialize  in  contract  prices,  timeliness,  access,  and 
manufacturing  efficiency,  all  of  which  may  be  lost  to  the 
contractor  and  the  user  if  MILSPECS  and  MILSTDS  are  required 
in  product  design,  development,  and  manufacturing. 

The  challenge  for  NDI  'PMs  is  to  leave  the  safety  and 
security  of  MILSPECS  and  MILSTDS  and  to  use  CIDs  and 
commercial  standards.  A  NDI  acquisition  strategy  encourages 
acquisition  agencies  to  break  inclinations  to  "cling  to  the 
security  of  MILSPECS  and  MILSTDS"  and  instead  to  take 
advantage  of  commercial  technology  development,  innovation, 
and  streamlining  initiatives.  The  PM' s  risk  is  one  of 
accepting  and  using  unfamiliar  commercial  design  and 
development  parameters.  With  commercial  S&S  come  assumptions 
about  the  conditions  and  tasks  under  which  the  product  must 
perform  as  well  as  the  intended  purpose  of  the  standard.   The 
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S&S  may  be  unique  to  a  particular  company,  product,  or  process 
and  not  easily  transferrable. 

The  specifications  and  standards  risks  consist  of 
either  choking  the  NDI  process  with  required  MILSPECS  or 
relying  on  CIDs  for  products,  parts,  and  spares  which  may  or 
may  not  match  MILSPEC  and  MILSTD  detail.  The  implicit 
understanding  of  combat  environments  and  conditions  may  also 
be  lost.  The  grasp  of  military  operational  conditions  in 
commercially- oriented  S&S  may  be  foregone.  Competitive 
reprocurements  that  depend  on  CIDs  as  technical  data  package 
references  may  jeopardize  functional  and  technical  performance 
in  the  field.  This  in  turn  depends  on  the  PM  being  equipped 
with  a  detailed  market  analysis  that  examines  the  suitability 
of  CIDs  to  meet  user  and  system*  requirements. 

A  shrinking  DTIB  is  one  trend  DOD  witnessed  after 
winning  the  Cold  War.  Stemming  this  trend  is  potential  dual- 
use  (military  and  civilian)  item  applications.  Defense- 
commercial  product  usage  enables  more  flexibility  in 
procurements,  closes  the  gap  between  military  and  commercial 
products,  and  keeps  the  DTIB  competitive  in  the  global 
marketplace  (Gansler,  p.  115,  1993)  .  Yet  the  defense 
industry,  like  the  acquisition  workforce,  is  resistant  to 
these  opportunities  (Mandel,  p.  193,  1994).  Dual-use  and 
commercial  specifications  in  this  post -Cold  War  era  are 
becoming  more  suitable  for  meeting  the  level  of  detail 
required  for  military  requirements.  But  despite  their 
political  popularity,  commercial  market  practices, 
specifications,  and  production  standards  continue  to  be  a  hard 
sell  (Schrage,  p.  B3 ,  1994). 
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d.  Test  and  Evaluation 

Test:  and  evaluation  (T&E)  programs  for  NDIs  must 
achieve  the  same  purpose  as  developmental  T&E  programs:  to 
define  risks.  This  risk  functions  to  consider  how  a  system 
meets  its  functional  specifications  in  the  intended 
operational  environment.  The  T&E  risks  in  NDI  acquisitions 
are:  testing  away  NDI  time  and  cost  savings  for  the  sake  of 
testing  and  risk  elimination;  incomplete  or  inadequate  design, 
development,  production,  and  logistics  support  T&E  by  both  the 
contractor  and  PM  due  to  the  pressures  of  an  accelerated 
acquisition  (Cain,  p.  24,  1993);  or  assuming  the  contractor's 
T&E  results  are  sufficient  and  merely  conducting  technical 
reviews  of  T&E  data  (Garcia,  p.  20,  1991)  .  Ineffective 
verification  of  compliance  with  requirements  and 
specifications  may  result.  For  example,  prototypes  may  be 
proposed  as  competitive  designs  without  sufficient  production 
and  support  facilities  and  activities.  PMs  who  implement 
streamlined  NDI  T&E  programs  and  use  T&E  data  reviews  in  place 
of  production  and  support  capability  evaluations  (or  at  least 
demonstrations)  may  be  creating  risks  in  their  programs.  NDI 
T&E  risks  therefore,  can  contribute  to  technical  performance 
and  supportability  risk  in  the  program. 

NDI  T&E  programs  are  a  form  of  risk  management,  not 
risk  elimination,  used  to  prove  technical  capabilities, 
operational  effectiveness,  and  suitability.  The  T&E  community 
sees  four  areas  of  risk  in  NDI  system  testing: 

1.  Requirements  may  not  be  clearly  understood. 


2 .  Definition  of  the  operational  environment  may  not  be 
clear  and  thus  can't  be  tested  with  a  level  of 
confidence. 
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3 .  Definition  of  system  interfaces  and  operability 
between  the  NDI  system  and  others  it  must 
interface/operate  with  may  not  be  clear. 


4 .  Test  documentation  and  support  issues  may  not  be 
addressed  in  the  test  program. 

Two  impediments  in  the  T&E  community  also  exist: 

1.  The  T&E  "mindset".  The  testers  orient  on  traditional 
developmental  approaches  and  ignore  non- traditional, 
NDI  T&E  approaches  tailored  to  a  streamlined 
acquisition  strategy.  This  non- traditional  approach 
may  require  different  .T&E  methods,  timelines,  and 
levels  of  detail. 


2 .  Lack  of  knowledge  and  experience  with  commercial  test 
practices  and  standards. 

A  streamlined  NDI  T&E  program  may  introduce  risks  of 
insufficient  source  evaluation,  inadequate  competition,  and 
incomplete  market  analysis.  This  may  lead  the  PM  to  overlook 
promising  alternatives. 

e.  Integrated  Logistics  Support 

"ILS  in  an  NDI  acquisition  can  be  more  difficult 
because  of  the  difference  in  an  acquisition  process"  (DSMC  NDI 
Acquisition  Guide,  p.  41,  1992) .  NDI  integrated  logistics 
support  (ILS)  activities  performed  in  DEMVAL  and  EMD  are  often 
accelerated  to  ensure  support  arrives  concurrently  with  the 
item.  ILS  risks  involve  events  that  disrupt  maintenance 
planning,  manpower,  supply  support,  support  equipment, 
technical  data,  training  and  training  support,  mission 
critical  computer  requirements,  packaging/handling/storage/ 
transportation,  facilities,  and  design  interface  of 
operational  capability  with  resources.  The  ILS  issue  becomes 
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one  of  identifying  what  trade-offs,  logistics-related 
requirements,  and  departures  from  developmental  ILS  are 
necessary. 

A  NDI  acquisition  strategy  creates  several  ILS 
risks.  While  real  reliability  and  training  data  may  be  in 
place,  reduced  lead  time  means  less  ILS  planning  time  and  time 
to  develop  organic  support.  The  ILS  risk  of  proliferating 
hardware  and  software  changes  (e.g.  when  DOD  is  unable  to  buy 
the  technical  data  rights  for  maintaining  configuration 
control  of  commercial  items)  can  be  significant.  ILS  risks  can 
contribute  tremendous  complications  to  NDI  acquisition 
planning  and  execution. 

A  PM  develops  an  ILS  plan  (ILSP)  in  conjunction  with 
the  NDI  acquisition  strategy.   This  outlines  risk  management 
measures  for  supportability  risks  specified  in  the  ILSP.   A 
NDI  program's  supportability  objectives  act  as  part  of 
functional  requirements.  Supportability  risks  may  appear  in 
design  by  overlooking  realistic  operational  conditions,  RAM, 
survivability,    and   test   equipment   required.      NDI 
supportability  plans  may  not  be  included  in  Government 
requests  for  contractor  proposals  (RFPs)  ,  bidding  contractors' 
responses,  and  as  NDI  source  selection  criteria.  ILS  risks  in 
NDI  competitive  proposals  may  include  neglect  of  warranties, 
data  rights,  and  best  value  procurements. 

The  type  and  level  of  support  affects  the  severity 
of  impacts  ILS  risks  have  in  an  NDI  acquisition.  The  user  and 
PM  may  not  clearly  decide  whether  to  use  organic  support, 
contractor  support,  or  a  mix  of  both  in  the  ILSP  as  interim 
and  longterm  measures.  Trade-offs  may  not  be  included  in  this 
decision.  The  decision  should  be  incorporated  into  the 
overall  acquisition  strategy.  Decisions  on  the  level  of 
repair  affects  spares  and  repair  parts  requirements.  The 
sources  of  supply  are  also  at  issue. 

The  rapid  availability  of  NDIs  might  outpace  their 
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logistics  support  structure  in  the  field.  An  accelerated 
acquisition  cycle  may  overpower  training,  maintenance,  and 
logistics  support  activities  for  a  new  item.  The  NDI  product 
may  lack  sufficient  detail  and  realism  in  commercial  training 
support,  training  equipment  and  documentation.  Test  and 
evaluation  data  on  design,  performance,  and  logistics  support 
may  not  be  complete  or  available.  Supply  items  may  lack  the 
necessary  lead  time  to  support  fielding  and  new  equipment 
training.  Reduced  commonality  and  standardization  accompanied 
by  increased  manpower  and  support  requirements  may  actually 
add  costs  over  the  product's  lifecycle  (Vandeviere,  p.  15, 
1994) .  Configuration  control  may  be  complicated  by  rapid 
obsolescence  (Cain,  p. 24,  1993).  This  then  jeopardizes 
component  integration  and  interoperability  as  components  and 
parts  are  repeatedly  being  replaced.  The  supplier  may  cease 
production  of  the  item  which  creates  a  problem  for  spares 
availability  and  proliferation  of  models.  This  reduces 
material  readiness  by  creating  incompatibility  of  system 
components.  The  item  may  require  contractor  maintenance 
support  in  an  operational  environment. 

To  summarize,  NDI  requirements,  acquisition  management 
environment,  S&S,  T&E,  and  ILS  risk  issues  will  challenge 
users,  PMs,  and  potential  contractors.  The  challenge  will  be 
to  balance  benefits  in  NDI  acquisition  costs  and  schedules 
with  technical  performance,  programmatic,  and  supportability 
risks.  Common  threads  tying  NDI  program  objectives  and  risks 
together  will  be  teaming  among  the  acquisition  community 
members,  addressing  traditional  biases,  market  analysis,  the 
clarity  and  stability  of  user  requirements,  and  the  ability  of 
the  PM  to  cope  with  the  accelerated  features  of  a  NDI 
acquisition  strategy. 
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E.   NDI  ACQUISITION  POLICY 

The  risk  management  portions  of  DOD  and  U.S.  Army 
acquisition  policy  and  instruction  are  thorough.  The  NDI 
sections  are  treated  likewise. 

1.  DOD  DIRECTIVE  5000.1 

DODD  5000.1  "requires  the  use  of  an  existing  U.S.  or 
allied  military  or  commercial  system  be  assessed  and 
thoroughly  reviewed  as  an  approach  to  satisfy  a  need  or 
requirement.  The  directive  also  states  that  when  tailoring  an 
acquisition  strategy  to  meet  individual  circumstances, 
consideration  should  be  given,  whenever  possible  and 
appropriate  to  maximizing  practical  use  of  commercial  "off- 
the-shelf"  (COTS)  products"  (PM  Notebook,  p. 1.5. 1-1,  1992). 

2.  Government  Policy 

The  National  Defense  Authorization  Act  of  1987  "requires 
DOD  to  use  NDI  to  fulfill  needs  to  greatest  extent  possible. 
It  also  requires  that  DOD  state  its  need  in  terms  of  functions 
to  be  performed,  performance  required,  or  essential 
characteristics"  (PM  Notebook,  p. 1.5. 1-1,  1992).  This  bill 
followed  the  1986  Packard  Commission's  report  which  proposed 
that  (President's  Blue  Ribbon  Commission  on  Defense 
Management,  19  86) : 

DOD  should  make  greater  use  of  components,  systems, 
and  services  available  off-the-shelf.  It  should 
develop  new  or  custom-made  items  only  when  it  has 
been  clearly  established  that  those  readily 
available  are  clearly  inadequate  to  meet  military 
requirements . 
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The  commission's  recommendation  to  the  Under  Secretary  of 
Defense  for  Acquisition  was  to  require  PMs  "to  receive  a 
waiver  before  using  a  product  made  to  military  specifications 
if  there  is  an  available  commercial  counterpart". 


F.   IMPLICATIONS  OF  RISK  MANAGEMENT  POLICY  FOR  NDI  ACQUISITION 
PROGRAMS 


The  paths  of  risk  management  policy  and  NDI  policy  do  not 
cross.  DOD  policy  emphasizes  both  risk  management  and  NDI  but 
in  differing  directions.  Risk  management  policy  in  the  5000.- 
series  keys  on  developmental  items  and  acquisition  strategies. 
NDI  policy  focuses  on  its  risk  mitigation  features  gained 
through  use  of  established  commercial  alternatives.  Cost  and 
schedule  benefits  aside,  no  description  or  instruction  for 
risk  management  in  an  NDI  program  appears  in  the  5000 . -series . 
Risk,  risk  management,  and  NDI  subject  area  searches  using  the 
Air  Force  Acquisition  Model  (AFAM)  supplement  1.4  dated  1 
December  1993  revealed  no  matches  between  the  three  subjects. 

This  policy  void  exists  within  the  expectation  that  the 
PM  will  "tailor"  the  risk  management  approach  to  NDI  much  like 
the  acquisition  cycle  is  tailored  to  NDI.  Tailoring  is 
accepted  as  overlapping,  combining,  or  omitting  elements  of 
the  acquisition  process  (DODI  5000.2,  p. 2-6,  1994).  A  NDI 
acquisition  strategy  is  a  tailored  version  of  the 
developmental  acquisition  process  according  to  the  type  of 
program,  program  risks,  user  requirements,  and  needs. 
Tailoring  NDI  acquisitions  according  to  program  risks, 
however,  is  not  accompanied  by  risk  management  guidance  or 
instructions  in  either  DODD  5000.1  or  DODI  5000.2. 

NDI  cost,  schedule,  performance,  supportability,  and 
programmatic  objectives  entail  certain  opportunities  and 
risks.   Today,  these  risks  must  be  managed  without  official 
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guidelines  because  DOD  and  the  Army,  though  quick  to  embrace 
NDI  benefits,  have  not  followed  their  enthusiasm  with  updated 
policy.  Dual -use  technologies,  NDIs,  and  COTS  items  are 
popular,  in  fact  mandated,  as  the  first  alternative  for 
filling  user  requirements.  Policy  is  needed  that  recognizes 
the  increased  use  of  commercial  technologies  for  military 
applications  and  their  corresponding  risks. 

The  risks  in  NDI  acquisition  are  not  as  widely  described 
or  documented  as  the  benefits.  The  absence  of  documentation 
and  description  is  due  to  lack  of  familiarity  with  the  NDI 
acquisition  risks  in  competitive,  commercial  (defense  and  non- 
defense  related)  industries  by  PMs,  users,  and  DOD  policy 
writers.  These  commercial  market  industries  provide  DOD  with 
a  growing  number  of  NDI  systems  not  initially  developed  to 
meet  a  military  need.  The  acquisition  community's  lack  of 
familiarity  means  a  lack  of  comfort  or  confidence  in 
commercial  practices  and  standards.  This  then  creates 
mistrust  and  inhibits  defense -commercial  interaction.  Without 
formal  risk  management  policy  and  guideline  initiatives  by  DOD 
acquisition  participants,  definitive  policy  and  instruction 
has  insufficient  support  or  momentum  to  be  officially 
implemented. 

In  NDI  programs,  probabilities  of  adverse  events  and 
their  consequences  in  requirements,  the  acquisition  management 
environment,  S&S,  T&E,  and  ILS  significantly  impact  the 
achievement  of  program  objectives.  PMs  need  more  explicit 
guidance  in  policy  and  instruction  regarding  NDI  risk 
management  in  the  streamlined,  accelerated  NDI  environment. 

1.  NDI  Requirements.  Policy  does  not  address  how 
requirements  generation  and  formulation  during  the  NDI  Concept 
Exploration  and  Definition  phase  creates  risk  and  how  it 
should  be  managed.  Requirements  and  expected  capabilities  are 
not  compared  to  what  is  realistic  and  commercially  available. 
Market  analysis,   investigation,  and  surveillance  are  not 
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adequately  considered  as  part  of  the  risk  management  process. 

2.  Acquisition  Management  in  the  NDI  Environment.  The 
claim  "this  is  the  way  it's  always  done"  captures  the  typical 
resistance  inherent  in  the  developmental  perspective  often 
used  by  PMs ,  users ,  and  DTIB  contractors .  Risk  management 
policy  describes  none  of  the  interrelationships  between  the 
acquisition  environment  and  its  participants:  PM,  user,  DOD, 
commercial  vendors  of  NDIs,  and  Congress.  Constrained 
resources,  competitive  pressures,  extensive  oversight,  and 
constant  change  create  continuous  friction.  These  frictions 
can  make  it  difficult  for  DOD's  PMs,  users  and  commercial 
representatives  to  work  together  effectively  toward  evaluating 
trade-offs  and  satisfying  user  needs.  The  milestone  reviews 
and  in- progress  reviews  of  a  NDI  program  lack  risk  management 
guidelines  for  examining  risk  management  effectiveness. 
Measures  of  effectiveness  and  . measures  of  performance  are 
absent  for  NDI  risk  management. 

3.  NDI  Specifications  and  Standards.  CIDs  are 
unfamiliar  ground  to  much  of  the  acquisition  workforce. 
MILSPECS  and  MILSTDS  provide  a  "comfort  level"  to  PMs  telling 
the  contractor  specifically  what  to  do  and  how  to  do  it. 
Managing  risks  inherent  in  the  use  of  CIDs  as  opposed  to 
MILSPECS  and  MILSTDS  is  an  unknown  because  extensive 
consultation  with  DTIB  manufacturers  on  technical  performance 
is  relatively  new.  Existing  risk  management  policy  does  not 
accommodate  well  the  loud  voice  that  the  DTIB  must  have  in 
establishing  performance  S&S  for  NDI  items.  To  the  contrary, 
under  existing  policies,  DOD  and  its  PMs  have  grown  accustomed 
to  being  the  dominant  voice  in  such  matters. 

4.  NDI  Test  and  Evaluation.  T&E  is  a  part  of  risk 
management,  specifically  risk  assessment  and  analysis.  Risk 
management  for  NDI  T&E  risks  overlaps  with  requirements,  the 
NDI  acquisition  environment,  and  specifications  risks.  DOD 
policy  does  not  spell  out  guidelines  for  managing  risks  in  NDI 
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testing  methodology,  scheduling,  or  integration  with  other 
systems  in  operational  environments. 

5.  NDI  Integrated  Logistics  Support  Planning.  The 
greatest  number  of  risks  in  fielding  NDI  involve 
supportability .  Risk  management  policy  neglects  the  market 
analysis  of  modification  requirements,  sustainment  of 
logistics  and  supply,  and  availability  of  the  product  and  its 
production  operations  over  the  typical  15-20  year  defense 
horizon.  DOD  policy  overlooks  risks  in  contractor  versus 
Army- run  depots,  buying  upgrade/pre-planned  product 
improvement  (P3I)  packages,  spares,  and  technical  data 
packages.  The  "other  sources  of  challenges  are  the  standard 
internal  DOD  processes  which  must  be  expedited  or  tailored  to 
accommodate  an  NDI  strategy"  (Vandeviere,  p. 15,  1994).  This 
policy  gap  must  deal  with  such  issues  as  training  development, 
logistics  and  maintenance  support,  personnel  management 
systems,  and  military  force  structure  planning. 

6.   SUMMARY 

DOD  and  U.S.  Army  risk  management  policy  provide  no 
explicit  details  or  guidance  on  managing  NDI -related  risks. 
DOD's  shortfalls  were  described  above.  Similarly,  the  Army 
has  not  addressed  the  growing  NDI  area  with  risk  management 
guidelines  with  which  a  PM  could  structure  an  NDI  risk 
management  program.  With  the  increasing  level  of  commercial 
products  in  DOD  and  U.S.  Army  inventories,  it  behooves  them  to 
initiate  policy  and  guidelines  for  NDI  risk  management 
programs . 

The  following  chapter  studies  the  efforts  of  the  Army's 
FAAD  Sensors  Program  Office  to  manage  the  risks  in  its  NDI 
acquisitions  and  assemble  a  risk  management  program  in  the 
absence  of  NDI  risk  management  policy. 
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IV.       THE    FAAD    SENSORS    CASE 

A.  PURPOSE 

The  purpose  of  this  chapter  is  to  examine  the  risk 
management  program  of  an  Air  Defense  Artillery  NDI  acquisition 
program  office.  The  chapter  begins  with  a  review  of  the  ADA'S 
critical  early  warning  radar  need  upon  retirement  of  the 
Forward  Area  Alerting  Radar  (FAAR) .  The  review  briefly 
describes  air  defense  doctrine  and  early  warning  radar 
requirements.  The  chapter  then  introduces  the  Forward  Area 
Air  Defense  (FAAD)  Sensors  program  which  includes  the  FAAR's 
NDI  replacements,  the  Ground  Based  Sensor  (GBS)  and  the  Light 
and  Special  Division  Interim  Sensor  (LSDIS) .  It  explores  the 
risks  facing  the  two  systems.  The  chapter  summarizes  the  PM' s 
overall  risk  management  program  and  risk  management  plan.  The 
chapter  considers  how  the  programs'  risks  and  risk  management 
may  have  resulted  in  the  delays  in  fielding  early  warning 
radars  and  the  consequent  ADA  force  readiness  gap.  The 
chapter  concludes  with  a  look  at  the  support  provided  by  DOD 
risk  management  policy  to  NDI  system  PMs  as  they  structure 
their  risk  management  programs. 

B.  EARLY  WARNING  OVERVIEW  AND  FAAD  ACQUISITION  IMPLICATIONS 

1.   The  FAAR  and  FAAD  Doctrine 

Early  warning  networks  (EWN)  support  battlefield  command, 
control,  communications,  and  intelligence  decisions  for  FAAD 
battalion  commanders  and  Army  division  commanders.  Early 
warning  radars  making  up  the  EWN  provide  continuous 
surveillance,  detection,  acquisition,  and  tracking  information 
on  potential  air  threats  flying  over  an  Army  division's  area 
of  operation.  Ideally,  the  radars  operate  both  day  and  night, 
in  all  weather  conditions,  in  both  electronic  countermeasures 
(ECM)  and  anti- radiation  missile   (ARM)  environments   (GBS 
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Acquisition  Plan,  p. 8,  1990). 

Until  1990,  the  Army's  ADA  branch  relied  on  the  AN/MPQ-54 
FAAR  to  provide  its  critical  early  warning  information.  The 
FAAR  early  warning  system  consisted  of  an  early  warning  van 
with  a  telescoping  radar  pulled'by  its  prime  mover,  the  Gamma 
Goat.  Air  intelligence  preparation  of  the  battlefield  (IPB) 
and  proactive  air  defense  coverage  depended  on  the  FAAR  to 
continuously  broadcast  early  warning  of  approaching  threat, 
unknown,  and  friendly  aircraft.  Without  inherent  early 
warning  capabilities  in  ADA  battalions,  the  battalions'  IPB 
processes  and  fire  units'  combat  effectiveness  suffer.  The 
result  of  this  lower  effectiveness  is  a  higher  attrition  rate 
of  prioritized  divisional  assets  by  enemy  air  threats  as  well 
as  increased  incidents  of  fratricide. 

The  Army  removed  the  FAAR  from  active  service  as  of  31 
October  1990  (LSDIS  Acquisition  Plan,  p.  7,  1990) .  The  FAAR 
was  scrapped  because  it  proved  highly  susceptible  to  ECM, 
suffered  from  a  poor  operational  readiness  rate,  and  did  not 
cue  ADA  fire  units  effectively  to  approaching  enemy  aircraft 
(Stolt,  June  1994).  The  FAAR's  Gamma  Goat  prime  mover 
experienced  consistent  maneuverability  limitations.  The 
radar's  range  limitations  impaired  its  survivability  (in  the 
face  of  ARM)  .  The  FAAR  system  had  incompatible  communications 
configurations  for  digital  radio  transmissions  and  its  design 
featured  no  air-droppable  capability  (Wilson,  August  1994) . 

No  field- ready  early  warning  replacement  systems  existed 
for  Desert  Shield  and  Desert  Storm.  Divisional  ADA  battalions 
deploying  to  the  Persian  Gulf  either  removed  their  FAARs  from 
"mothballs"  for  minimal  early  warning  capability  or  deployed 
without  them.  By  the  conclusion  of  the  campaign,  divisional 
ADA  units  would  be  relying  on  "1940' s  Eyeball  Technology", 
commonly  called  binoculars,  as  their  organic  EWN  capability 
(Tremmel,  1994)  . 
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Replacing  binoculars  with  effective  early  warning  radar 
technology  has  become  an  urgent  requirement  for  the  ADA  branch 
and  the  Army  (FAAD  Sensors  PMO,  1994)  .  ADA  FAAD  doctrine 
depends  on  an  effective  early  warning  component  to  succeed. 
FAAD  doctrine  integrates  command  and  control  nodes,  fire  units 
such  as  Avenger  teams  and  Bradley  Stinger  Fighting  Vehicle, 
and  early  warning  radars  to  provide  comprehensive  FAAD 
protection  to  forward  deployed  Army  divisions.  DOD  and  the 
Army  tolerate  this  EWN  readiness  gap  while  replacement  radars 
are  tested  and  produced  (Tremmel,  1994).  The  readiness  gap 
left  by  the  retirement  of  the  FAAR  mandated  an  effective 
defense  acquisition  program  to  fill  the  early  warning  need  and 
make  FAAD  doctrine  viable  (Tremmel,  1994). 

2.   Acquisition  Implications 

Past  FAAD  acquisition  program  difficulties  and  the  FAAR's 
problems  have  created  an  environment  of  high  risk  for  future 
programs  and  close  DOD  scrutiny  for  ADA  branch  PMs .  FAAD 
acquisition  programs  suffered  a  string  of  disappointments  in 
the  1980' s  and  early  1990' s.  The  Divisional  Air  Defense  Gun 
(the  Sergeant  York) ,  the  Roland,  the  Air  Defense  Ant i- Tank 
System,  and  the  Fibre-Optic  Guided  Missile  were  all  FAAD- 
related  programs  that  were  cancelled  due  to  inadequate 
technical  performance  or  high  design  and  development  costs. 

The  FAAD  programs'  reputation  for  expensive  failures 
combined  with  the  FAAR's  lifecycle  maintenance  and 
supportability  woes  increases  the  danger  of  performance, 
programmatic,  and  supportability  risks.  Any  future  ADA 
acquisition  to  support  FAAD,  whether  a  weapon  system  or  radar, 
would  also  require  close  management  of  its  cost  and  schedule 
risks.  This  environment  would  dictate  more  substantial  and 
detailed  FAAD  Sensors  program  risk  management. 
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A  lower- risk  acquisition  of  EWN  systems  requires  that  the 
PM  recognize  the  Desert  Storm' s* legacy.  This  legacy  consists 
of  the  expectations  by  DOD,  Army,  and  Congressional 
policymakers  that  the  U.S.  Air  Force  (as  it  did  in  the  Gulf 
War)  will  always  maintain  air  superiority  in  any  conflict. 
The  consequence  of  this  assumption  is  that  approval  of  major 
defense  acquisition  programs  and  resource  decisions  would 
neglect  or  discount  air  defense  early  warning  as  a  justifiable 
need  that  required  and  deserved  funding.  Given  past  FAAD 
acquisition  failures,  their  inadequate  risk  management 
efforts,  and  arguments  against  the  need  for  Army  EWN,  close 
risk  management  is  needed  to  prevent  loss  of  program  support 
and  to  control  possible  environmental  (political)  risks. 

This  set  of  circumstances  demanded  that  a  successful  EWN 
acquisition  program  had  to  field  long-term,  operationally 
effective,  and  suitable  systems  while  effectively  managing 
risk.  A  streamlined  and  tailored  NDI  system  acquisition  which 
met  operational  requirements  and  featured  limited  program 
risks  would  be  ideal  for  the  situation. 

C.   TEE  GROUND  BASED  SENSOR  (GBS) 

1.   GBS  System  Overview 

The  FAAD  Sensors  PM  manages  the  GBS  as  one  of  two  systems 
acquired  to  provide  ADA  its  EWN  capabilities.  The  AN/MPQ-64 
GBS  is  a  three-dimensional  battlefield  air  defense  radar  which 
employs  a  modern,  phased  array  antenna  for  azimuth,  elevation, 
and  range  data  for  approaching  air  threats.  It  automatically 
detects,  tracks,  identifies,  and  reports  targets.  It  tracks 
rotary  and  fixed- wing  aircraft,  unmanned  aerial  vehicles 
(UAVs) ,  and  cruise  missiles.  It  classifies  targets  as  fixed- 
wing  or  rotary  aircraft  and  differentiates  air  targets  from 
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ground  targets.  It  provides  cueing  data  and  track  updates 
once  every  two  seconds  using  digital  communications  to  fire 
units.  Its  40km  range  allows  it  to  facilitate  target 
acquisition  beyond  ordnance  release  lines  to  maximize  fire 
unit  reaction  time  and  engagement  ranges.  Its  Identification 
Friend  or  Foe  (IFF)  feature  prevents  fratricide  by 
interrogating  aircraft  in  terms  of  their  coded  identity.  It 
tracks  and  searches  for  targets  simultaneously  (GBS  Integrated 
Program  Summary  (IPS) ,  p.  G-l,  1994) . 

The  GBS  is  a  NDI  system.  It  is  the  AN/TPQ-36A  Firefinder 
radar  modified  for  FAAD  missions  and  requirements  (see  Figure 
4) .  It  is  manufactured  by  Hughes  Aircraft  Corporation.  The 
GBS  features  a  proven  system  capability  with  extensive  data 
and  support  information  concerning  cost,  schedule, 
performance,  and  supportability  program  objectives. 

2.   The  GBS  Acquisition  Strategy 

The  GBS  mission  need  studies  began  in  1979  as  part  of  an 
automation-based,  ADA  force  modernization  initiative.  FAAD 
doctrine  and  a  GBS  acquisition  strategy  evolved  from  these 
studies.  The  acquisition  strategy  emphasizes  systems  near  or 
ready  for  production.  The  requirement  for  a  FAAR  replacement 
was  approved  by  the  Vice  Chief  of  Staff,  Army  on  3  September 
1985.  The  requirement  established  that  fielding  begin  in 
Fiscal  Year  1991  employing  an  NDI  sensor  and  common  Army 
hardware  and  software.  The  Milestone  III  Acquisition  Decision 
Memorandum  approved  the  GBS  NDI  Acquisition  Strategy  on  19 
August  1986  (GBS  IPS,  p.  C-l,  1994) . 

The  FAAD  GBS  Request  for  Proposal  (RFP)  was  issued  on  19 
April  1988.  Hughes  was  the  only  respondent.  Candidate 
evaluation  tests  proved  its  proposed  AN/TPQ-3  6A  radar  did  not 
meet  requirements.  The  solicitation  was  terminated  on  the 
basis  of  unattainable  performance  requirements  contained  in 
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Figure  4  The  Ground  Based  Sensor  (Source:  FAAD  Sensors 
PMO,  1994) 


the  ORD.   Minimum  operational  requirements  were  reassessed, 
the  GBS  ORD  was  amended,  and  the  new  ORD  approved  by  the  Army 
on  29  November  1989. 

With  relaxed  requirements,  the  Army  issued  a  competitive 
resolicitation  for  GBS  development  contract  bidders  which 
included  a  Source  Selection  Evaluation  Test  (SSET) .  The  SSET 
consisted  of  proposal  evaluations  and  field  tests  focusing  on 
both  technical  and  operational  requirements.  The  proposal 
emphasized  that  the  Government  desired  the  "best  value"  GBS 
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system  to  satisfy  its  requirements  (GBS  IPS,  p.  C-2,  1994) 4. 

Seven  candidates  responded  to  the  new  SSET  contract 
solicitation.  The  goal  was  a  NDI  sensor  with  reduced 
Government  development  costs  and  a  shorter  development  phase. 
Hughes  won  the  competitive  evaluation  and  was  announced  as  the 
contract  awardee  in  December  1991. 

GBS  development  began  on  February  1992.  Pre-production 
sensors  were  delivered  and  pre-production  qualification  tests 
(PQTs)  were  successfully  completed  as  part  of  the  program's 
Phase  I.  The  initial  sensors  met  performance  and  operational 
requirements  in  accordance  with  exit  criteria.  Phase  II 
testing,  which  concerns  the  integration  of  GBS  with  FAAD 
command,  control  and  intelligence  (FAADC3I)  systems,  Avenger, 
and  Man  Portable  Air  Defense  Stinger  (MANPADS)  teams  will 
continue  until  pre-production  sensors  are  completed  and 
delivered  in  February  1996.  GBS  fielding  will  finally  begin 
in  1997  (GBS  IPS,  p.  C-4,  1994). 

D.   THE  LIGHT  AND  SPECIAL  DIVISION  INTERIM  SENSOR  (LSDIS) 
1.   LSDIS  System  Overview 

The  FAAD  Sensors  PM  manages  the  LSDIS  as  the  second  of 
the  two  systems  being  acquired  for  FAAD  EWN  capabilities. 
Whereas  the  GBS  is  initially  designated  for  ADA  units 
supporting  armored,  mechanized,  and  armored  cavalry  divisions, 


4  Note:  Best  value  is  defined  as  "acquiring  a  production 
ready  system  through  full  and  open  competition  that  best  meets 
the  Army's  needs  considering  cost,  schedule,  performance  and 
risks"  (LSDIS  Acquisition  Plan  (AP) ,  p.  8,  1990).  The  best 
value  approach  strives  to  meet  Army  needs  by  the  product 
itself  or  with  a  P3I  program.  It  differentiates  minimum 
requirements  from  objective  requirements.  The  contractor  can 
meet  minimum  requirements  rather  than  objective  ones.  The 
disadvantage  is  that  the  Government  is  not  assured  of  ever 
reaching  those  objective  requirements. 
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the  LSDIS  is  dedicated  primarily  to  ADA  battalions  in  light 
infantry,  airborne,  and  air  assault  divisions  and  contingency 
corps.  The  LSDIS  is  a  two-dimensional  air  defense  radar  that 
alerts  fire  units  with  azimuth  and  range  data  on  approaching 
short-range,  low- altitude  aircraft.  It  is  a  highly  mobile, 
lightweight,  manportable  system  which  detects  and 
automatically  classifies  fixed-wing  and  rotary  aircraft  up  to 
a  range  of  20km  (see  Figure  5)  .  It  is  an  interim  system  until 
a  light-weight  version  of  the  GBS  is  fielded  (no  earlier  than 
Fiscal  Year  00)  (LSDIS  Acquisition  Plan,  p.  7,  1990) . 

The  LSDIS  is  a  NDI  system.  It  is  intended  to  support 
rapid- deployment  light  and  special  divisions  in  low- intensity 
conflicts  with  early  warning  alerts  of  incoming  threat 
aircraft.  It  is  classified  as  an  urgent,  directed  procurement 
to  expedite  its  fielding  (LEWDD  Directed  Procurement 
Memorandum,  p.  1,  1990) . 

2.   The  LSDIS  Acquisition  Strategy 

The  LSDIS  NDI  acquisition  strategy  consists  of  a 
competitive  procurement  reflecting  the  FAAR's  retirement, 
pursuit  of  the  best  technology  available,  and  minimizing 
expected  cost  and  schedule.  The  directed  procurement  "label" 
originated  from  the  urgent  need  to  field  an  early  warning 
sensor  to  light  and  special  'division  ADA  units  until  a 
lightweight  version  of  GBS  could  be  deployed. 

The  LSDIS  acquisition  cycle  began  with  the  approval  of 
the  Lightweight  Early  Warning  Sensors  Required  Operational 
Capability  (the  LSDIS 's  original  ORD)  on  22  June  1990  and  the 
Directed  Procurement  of  the  Lightweight  Early  Warning 
Detection  Device  (LEWDD)  Memorandum  published  26  July  1990. 
These  were  based  on  the  technical  requirements  of  the  U.S. 
Marine  Corps  LEWDD  program.  The  directed  nature  of  the  LSDIS 
procurement  emphasized  an  accelerated  solicitation  and  source 
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Figure  5  The  Light  and  Special  Division  Interim  Sensor 

selection  process  (LSDIS  IPS,  p. 4,  1993). 

The  FAAD  Sensors  PM  recognized  that  not  all  system 
requirements  would  be  met.  This  was  particularly  true  with 
objective  requirements.  Proposal  evaluations  were  based  on 
best  value  and  considered  tradeoffs  between  objective 
performance  requirements  and  current  technical  capabilities 
(minimum  requirements),  cost,  and  schedule  (LSDIS  AP,  p. 10, 
1990)  . 

The  LSDIS  RFP  was  released  on  30  July  1990.  Lockheed 
Sanders  and  Lear  Siglar  were  the  only  bidders.  The  LSDIS 
program  emphasized  a  streamlined  NDI  approach  that  allowed  the 
Government  to  rapidly  meet  its  urgent  need.  The  competition 
emphasized  that  bidders  provide  reduced  cost,  efficient,  and 
"best  value"  systems.  Lockheed  Sanders  was  awarded  the  best 
value  contract  on  13  November  1990. 
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LSDIS  product  testing  began  after  the  contract  award  in 
May- June  1991.  The  LSDIS  contract  required  a  First  Article 
Test  (FAT)  and  a  PQT  after  which  additional  production 
contract  awards  would  be  issued.  The  date  of  the  first  unit 
(to  be)  equipped  (FUE)  was  late  1992  with  fielding  to  be 
complete  in  1994  (LSDIS  AP,  p. 7,  1990).  The  LSDIS  FAT  was 
successfully  completed  during  -1  July  1991-15  November  1992 
with  the  PM  conditionally  accepting  the  first  unit.  The  PQT 
occurred  from  2  March- 7  May  1992.  The  LSDIS  failed  to  meet 
operational  requirements  and  system  specifications  due  to 
false  target  rates,  reliability-availability-maintainability 
(RAM)  shortfalls,  and  azimuth  alignment  problems.  A  second 
PQT  initiated  on  7  June  1993  revealed  all  shortfalls  were 
corrected  and  no  further  operational  failures  were  exposed. 
The  contractor's  inability  to  meet  operational  requirements 
resulted  in  higher  production  costs  and  schedule  delays  of  one 
year  (LSDIS  IPS,  p. 10,  1993). 

The  LSDIS  NDI  acquisition  strategy  required  that 
production  begin  only  after  successful  PQT  completion.  The 
LSDIS  directed  acquisition  authorized  limiting  specification 
standards,  test  plans,  and  contract  data  requirements  in  its 
RFP  as  a  means  of  saving  costs.  Waivers  of  those  items 
designated  as  non-cost  effective  contract  requirements  were 
intended  to  shorten  the  program's  duration.  The  acquisition 
strategy  stressed  "streamlined"  program  planning,  testing,  and 
ILS  (LSDIS  IPS,  p. 10,  1993). 

E.   RISKS  AND  RISK  PERCEPTIONS  IN  THE  FAADS  SENSORS  PROGRAMS 

1.   6BS  Acquisition  Program 

a.  Technical  Risks 

The  revised  requirements  published  in  the  GBS 
resolicitation  reflected  the  Government's  best  value  approach. 
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New  minimum  operational  requirements  allowed  the  contractor  to 
make  tradeoffs  in  technical  and  performance  capabilities.  The 
GBS  design  was  simplified  as  a  two- (azimuth  and  range  only) 
rather  than  three-dimensional  radar  to  meet  cost  and  schedule 
objective  requirements  (Eison,  1994) .  Warranties  for 
reliability  and  performance  of  accepted  units  assured 
compliance  with  all  minimum  requirements.  The  PM  rated  GBS 
technical  risk  as  low  to  moderate  for  meeting  minimum 
requirements  (GBS  Acquisition  Plan,  p. 9,  1990). 

The  PM  rated  the  risk  of  meeting  technical  performance 
objective  requirements  broadly  as  being  low  to  high.  This 
rating  encompassed  the  known,  current  technical  performance  of 
the  radar  (based  on  minimum  requirements)  while  accounting  for 
unknowns  in  future,  P3I -based  radar  performance  (objective 
requirements) . 

Jb.   Cost  Risks 

The  PM  rated  the  GBS  cost  risk  as  low  to  moderate. 
The  GBS  design  was  based  on  the  Firefinder  configuration  which 
initially  reduces  the  development  requirements  and  costs. 
Schedule  and  performance  risks  are  also  dampened  by  the  mature 
design.  The  cost  risk  rating  considers  the  range  of  possible 
performance  capabilities  and  contracting  costs  resulting  from 
a  best  value  approach.  Cost  history  and  production  experience 
with  the  AN/TPQ-36A  Firefinder  indicate  low  cost  risks  in 
production.  The  GBS  PM  used  a  Firm- Fixed- Price  (FFP)  contract 
to  contain  Government  cost  risks.  The  PM's  conclusion  was 
that  a  NDI  acquisition  strategy  would  procure  the  best 
technology  available.  A  reasonable  contract  price  was 
expected  through  the  competitive  bidding  process  (GBS  AP,  p. 
10,  1990)  . 

The  PM  raised  the  GBS  cost  risk  to  moderate  when  the 
P3I  program  was  added.   These  cost  risks  reflected  research 
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and  development  (R&D)  and  design  modifications  required  to 
achieve  objective  performance  requirements.  They  also 
consider  a  different  contract  type.  Hughes  Aircraft 
Corporation  would  expect  a  Cost-*Plus  contract  to  cover  its  R&D 
costs  and  higher  risks  assumed  in  attaining  higher  objective 
performance  requirements.  If  Hughes  or  a  competitor  agreed  to 
a  FFP  contract  (given  that  some  P3I  developments  have  existing 
technological  solutions) ,  cost  risks  would  be  lower  for  the 
Government  and  moderate  (rather  than  high)  for  the  contractor. 
GBS's  objective  performance  requirements  would  be  pursued 
early  in  the  acquisition  cycle  if  additional  funding  were 
approved  for  the  GBS  P3I  program  (GBS  AP,  p.  11,  1990) . 

c.  Schedule  Risks 

The  PM  rated  the  GBS  schedule  risk  as  low  for 
meeting  a  two  year  delivery  schedule  for  a  first  production 
sensor  upon  contract  award.  Market  surveys  supported  this 
assessment.  Surveys  revealed  the  risks  of  late  fielding  due 
to  completion  of  a  P3I  program  increased  schedule  risks  to  a 
range  of  moderate  to  high.  The  PM  intended  to  pursue  mature 
technology  as  part  of  the  P3I.  Schedule  risks  involving  the 
completion  of  the  longer- range  P3I  program  depended  on  the 
increased  technological  complexity  of  the  sensor  and  expected 
costs  to  achieve  those  objective  performance  requirements  (GBS 
AP,  p. 11,  1990)  . 

d.  Supportability  Risks 

The  PM's  management  of  supportability  risks  is  based 
on  acquiring  a  validated  technical  data  package  (TDP) .  The 
purchase  of  a  TDP  is  intended  to  reduce  risks  concerning 
future  repair  parts  by  Government  ownership  of  technical 
documentation.  Future  repair  parts  procurements  could  then  be 
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competed  to  find  adequate  support  contractors  to  provide 
uninterrupted  parts  over  the  systems  entire  lifecycle  (GBS  AP, 
p.  11,   1990) .     No   risk   rating  was   assigned   to   system 
supportability  risks  in  the  GBS  Acquisition  Plan.   See  Figure 
6  for  the  GBS  Risk  Assessments. 

2.   LSDIS  Acquisition  Program 

a.  Technical  Risks 

The  PM  rated  the  LSDIS  technical  risk  as  low  to 
moderate.  As  with  the  GBS,  the  LSDIS  program  followed  a  best 
value  acquisition  approach.  This  approach  allowed  the  PM, 
armed  with  market  survey  results,  to  compete  the  procurement. 
However,  LSDIS  performance  would  be  tested  during  sensor 
production.  The  contractor  would  warranty  reliability  and  the 
fielded  systems'  performance  after  acceptance.  Risk 
assessments  would  be  based  on  FAT  and  PQT  results  (LSDIS  AP, 
p. 11,  1990) . 

Jb.   Cost  Risks 

The  PM  rated  cost  risk  as  low  based  on  the  LSDIS  FFP 
contract  and  competitive  procurement  strategy.  LSDIS  system 
configurations  represent  low  cost  risk  with  existing  cost 
history  and  production  background  already  established.  Price 
again  was  a  factor  in  the  best  value  approach  to  reduce  the 
Government's  cost  risks.  Price  considerations  under  a 
competitive  NDI  acquisition  strategy  was  not  anticipated  to 
raise  cost  risk  above  a  low  rating  (LSDIS  AP,  p. 11,  1990). 
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Figure  6  The  GBS  Risk  Assessments  (Source:  GBS  IPS,  1994) 
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c.  Schedule  Risks 

The  PM  rated  schedule  risk  as  being  moderate  based 
on  the  urgent  and  accelerated  schedule.  The  contractor  was 
expected  to  meet  LSDIS's  directed  procurement  fielding  dates 
one  year  after  contract  award  (LSDIS  AP,  p. 11,  1990). 

d.  SupportaJbillty  Risks 

No  description  of  LSDIS  supportability  risks  or  risk 
reduction  measures  is  in  the  LSDIS  AP.  Support  is  rated  as 
moderate  in  the  LSDIS  IPS  Risk  Assessment.  The  1990  LSDIS 
Light  Sensor  Market  Study  did  not  address  contractor  logistics 
support.   See  Figure  7  for  the  LSDIS  Risk  Assessments. 

F.   THE  FAAD  SENSORS  RISK  MANAGEMENT  PROGRAM 

1.   FAAD  Risk  Management  and  DOD  Policies 

The  FAAD  Sensors  PM  believes  NDI  acquisition  strategies 
equate  to  risk  mitigation  (FAAD  Sensors  PMO,  August  1994) . 
Both  the  GBS  and  LSDIS  risk  management  programs  demonstrate 
the  characteristics  described  by  DODD  5000.1  and  DODI  5000.2. 
Both  the  GBS  and  LSDIS  risk  management  programs  outlined 
performance,  cost,  and  schedule  risks.  Supportability  risks 
are  mentioned  in  passing.  The  FAAD  Sensors  PM  used  a 
streamlined  risk  identification,  assessment,  and  reduction 
process  as  a  risk  management  program.  The  GBS  and  LSDIS  risk 
identifications  and  ratings  and  their  rationale  are  simple  and 
direct. 

DODI  5000.2  does  not  explicitly  outline  a  NDI  risk 
management  program  for  NDI  PMs .  Army  Regulation  (AR)  70-1 
makes  no  explicit  connection  between  NDI  and  risk  management. 
No  quantified  or  qualified  risk  analysis  accompany  the  GBS  and 
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Figure  7  The  LSDIS  Risk  Assessments  (Source:  LSDIS  IPS,  1994) 
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LSDIS  risk  assessments  to  explain  how  they  were  obtained. 
Consideration  of  NDI- specific  risks  in  requirements, 
specifications,  T&E,  and  ILS  are"  not  evident.  This  is  because 
existing  policy  does  not  describe  how  specific  a  streamlined 
risk  management  approach  must  be,  let  alone  refer  to  any  NDI- 
specific  risks.  DOD  policy  provides  no  NDI  risk  management 
references  for  NDI  PMs  to  cite.  Due  to  this  lack  of  guidance, 
NDI  solutions  to  urgent  needs  and  force  readiness  gaps  can 
lead  to  incomplete  risk  management  programs.  PMs  may  tailor 
developmental  item  risk  management  guidelines  and  risk 
assessments  formats  found  in  DODI  5000.2  and  DOD  5000. 2 -M  to 
suit  NDI  program  needs.  However,  tailoring  developmental  risk 
management  policies  does  not  assure  successful  NDI  risk 
management . 

The  NDI  GBS  and  LSDIS  programs  constitute  tailored, 
streamlined  acquisitions  of  mature  technology  according  to  DOD 
and  Army  streamlining  guidance.  The  FAAD  Sensors  PM' s  risk 
management  practices  incorporate  tailoring  and  streamlining  in 
his  simplified  identify-assess-reduce  structure.  The  GBS 
acquisition  strategy  indicates  that  "the  Army  prefers  to  buy 
systems  already  designed,  developed,  tested,  and  in 
production,  or  at  least  where  principal  components  are  in 
production"  (GBS  IPS,  p.C-4,  1994) .  Any  risks  involved  with 
such  an  approach  are  not  covered  in  an  Army  policy  or 
memorandum,  despite  NDI's  popularity  in  Army  acquisitions. 

DOD  5000. 2  -M  does  not  outline  a  NDI  risk  assessment 
format.  The  GBS  and  LSDIS  risk  assessment  formats  are  broad 
but  follow  the  developmental  risk  assessment  structure  found 
in  DOD  5000. 2 -M.  Any  application  of  DOD  4245. 7-M  risk 
identification  and  assessment  procedures  to  the  indicated  GBS 
and  LSDIS  cost,  schedule,  performance,  and  supportability 
objectives  are  tailored  or  overlooked.  The  DSMC  Risk 
Management  Guide  and  the  AFAM  similarly  do  not  cover  NDI  risk 
management  policy,  plans,  or  risk  assessment  formats.   These 
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possible  risk  management  references  are  not  mentioned  in  the 
GBS  or  LSDIS  risk  management  documentation. 

2 .   The  GE5  Risk  Management  Program 

The  GBS  risk  management  program's  risk  descriptions  in 
its  acquisition  plan  and  risk  assessments  in  its  IPS  meet 
minimum  DOD  policy  requirements.  The  GBS  risk  management 
program  uses  the  DOD  5000. 2 -M  risk  assessment  format.  The 
GBS's  risk  management  program  meets  policy  requirements  by 
articulating  risk  planning,  assessment,  and  reduction 
measures. 

The  PM  implemented  technical  risk  reduction  by  using  a 
two-tiered  approach  to  technical  performance.  The  two  tiers 
are  minimum  and  objective  technical  performance.  A  system  had 
to  perform  within  this  range  to  meet  the  Army's  minimum 
requirements.  The  GBS  contract  bidders  would  then  be 
considered  as  qualified  potential  candidates.  The  PM' s  market 
surveys  indicated  low  risk  of  meeting  the  minimums.  Higher 
risk  requirements  were  designated  as  objective  or  "desired" 
requirements  rather  than  setting  them  as  minimums  as  in  the 
original  RFP.  Technical  risk  was  reduced  using  testing  during 
source  selection.  Production  contract  options  would  not  be 
exercised  unless  pre-production  systems  completed  technical 
tests  successfully.  These  measures  reduced  risks  in  fielding 
production  units  which  did  not  comply  with  warranties  on 
required  performance  levels  (GBS  AP,  p. 10,  1990) . 

To  meet  objective  or  "desired"  requirements  meant  the  PM 
had  received  funding  approved  for  a  GBS  P3I.  The  GBS  P3I 
program  is  currently  unfunded.  This  programmatic  uncertainty 
accounts  for  the  GBS  configuration  changes  to  the  HMMWV  prime 
mover  (low  risk) ,  radar  hardware  and  software  performance 
modifications  (moderate  risk) ,  and  future-based  radar 
modifications  to  support  the  GBS's  projected  role  as  a  "common 
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corps  sensor"  (GBS  AP,  p.  9,  1990). 

The  risk  of  publishing  unattainable  requirements  in  the 
original  solicitation  created  follow-on  risks  to  GBS 
performance,  cost,  schedule,  and  programmatic  objectives.  The 
sole  bidder  to  the  original  RFP,  Hughes,  did  not  meet  the  user 
requirements  during  candidate  system  testing.  Effective 
market  analysis  may  have  revealed  that  the  combination  of 
multiple  systems'  characteristics  stated  in  the  ORD  did  not 
exist  in  any  one  radar  configuration.  A  complete  market 
analysis  could  have  indicated  problems  with  the  original 
requirements  in  1985. 

The  new  ORD  and  resolicitation  of  the  GBS  contract  began 
four  years  later.  The  competition  for  contract  award  was  not 
completed  until  1991.  Six  years  in  schedule  were  added.  The 
program  management  costs  now  included  the  resolicitation  and 
re -evaluation  of  candidate  radars  based  on  a  new  ORD.  The  new 
schedule  extended  far  beyond  the  retirement  date  of  the  FAAR 
and  a  FAAD  EWN  capabilities  gap  developed.  The  GBS's 
objective  performance  requirements  now  hinge  on  unfunded 
product  improvements.  The  impact  is  seen  in  the  PM  being 
forced  to  trade  off  objective  performance  in  order  to  expedite 
fielding  a  radar  with  the  minimum  required  performance 
capabilities.  The  programmatic  risks  were  realized  when  DOD 
decreased  production  funding  and  the  number  of  systems  it 
would  buy  from  77  to  57  in  1994  (Department  of  the  Army  ADA 
Highlights  Memorandum,  p.  6,  June  1994). 

The  GBS  NDI  acquisition  strategy  emphasizes  the  benefits 
of  NDI  streamlining  described  earlier.  Although  minimum 
performance  requirements  were  met  during  SSET,  the  GBS  best 
value  procurement  did  not  meet  all  objective  performance 
requirements.  The  PM  and  user  accepted  technical  performance 
trade-offs.  These  decisions  concentrated  on  performance  that 
maximized  mission  support  of  FAADC3I  and  cueing  FAAD  weapon 
systems   rather   than  maximizing   GBS   radar   performance. 
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Deferred  performance  objectives  may  cause  delays  in  procuring 
GBS  P3I  longlead  items  and  eventually  result  in  a  production 
stoppage . 

The  problem  with  a  production  stoppage  is  that  resulting 
production  line  layoffs,  retraining,  facility  overhead  costs, 
and  start-up  costs  risk  further  schedule  delays  and  cost 
escalation  to  both  the  prime  and  subcontractors  (GBS  IPS,  p. 
C-7,  1994) .  The  traditionally  low- risk  NDI  cost  and  schedule 
features  have  also  contributed  to  greater  technical 
performance  and  programmatic  risks  for  the  GBS  program.  The 
PM  was  forced  to  "broad  brush"  the  technical  performance  risk 
rating  based  on  the  uncertainty  created  by  an  unfunded  P3I 
program. 

The  FAAD  Sensors  PM  currently  waits  for  additional 
funding  approval  to  incorporate  product  improvements  into  the 
GBS  configuration.  Without  the  improvements,  the  ADA  branch 
fields  a  minimum- capabilities  system,  delays  procurement  of 
long- lead  items,  and  faces  possible  production  shutdowns. 
With  the  improvements,  the  GBS  schedule  grows  again,  incurs 
more  costs  spread  over  fewer  systems,  and  ADA  endures  more 
pressure  to  sell  the  program  and  justify  its  need. 

The  PM  planned  to  reduce  GBS  cost  risks  based  on  the  type 
of  contract  awarded  for  both  production  and  P3I.  Pre- 
production  and  production  contracts  were  FFP  with  performance 
warranties.  The  contractor  assumed  the  cost  risks.  The 
Government  could  reject  options  that  were  too  risky.  A  Cost- 
Plus  Incentive  Fee  award  would  be  based  on  levels  of  proposed 
performance  compared  to  desired  requirements  (GBS  AP,  p.  11, 
1990)  .  Cost  containment  decisions  in  not  exercising  product 
improvement  options  and  the  use  of  a  FFP  contract  limited 
Government  risk  and  enhanced  program  support  in  the  short 
term. 

The  PM' s  schedule  risk  reduction  effort  relied  on  the 
best  value  NDI  acquisition  strategy.   Potential  contractors' 
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proposals  considered  production  and  delivery  schedules.  GBS 
schedule  risks  were  decreased  by  restricting  production  starts 
until  the  completion  of  the  pre-production  program.  This  risk 
reduction  measure  had  its  own  drawback,  however,  of  extending 
the  program's  length  over  an  unacceptable  period  of  time  (GBS 
AP,  p. 11,  1990) . 

Supportability  risks  were  assessed  with  limited  depth. 
The  Firefinder  radar  has  a  previously  established  support 
structure,  supply  and  parts  channels,  and  a  training  support 
package.  While  these  could  be  tailored  to  the  GBS  fielding, 
support,  parts  flow,  and  training  programs  must  be  oriented  to 
FAAD  missions  and  roles. 

Risk  minimization  by  the  FAAD  Sensors  Program  Office 
considered  other  components  besides  objectives.  The  best 
value  competitive  acquisition  strategy  incorporated  proposed 
contractor  risk  reduction  measures  into  the  source  selection 
evaluations  and  contract  proposal  evaluations.  The  PM' s  test 
program  includes  a  logistics  demonstration  in  conjunction  with 
operational  testing  to  verify  ILS  capabilities  and  support 
concepts. 

3 .   The  LSDIS  Risk  Management  Program 

The  LSDIS  acquisition  program's  risk  descriptions  in  its 
acquisition  plan  and  risk  assessments  in  its  IPS  meet  minimum 
DOD  policy  requirements.  The  LSDIS  risk  management  program 
generally  follows  the  DOD  5000.. 2 -M  risk  assessment  format. 
The  waiver  of  non- statutory  requirements  (due  to  its  urgent 
and  directed  procurement  classification)  reflects  the  more 
simplified  and  less  detailed  risk  assessment  structure. 

The  PM  employed  a  LSDIS  technical  risk  reduction  effort 
similar  to  the  GBS  program.  The  Army's  performance 
requirements  spanned  the  range  of  essential  requirements  and 
desired  capabilities.  Minimum  performance  requirements 
consisted  of  capabilities  and  characteristics  needed  by  the 
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user.  Capabilities  described  in  contractor  proposals  that 
exceeded  the  basic  requirements  were  considered  objective 
performance  requirements .  The  LSDIS  contract  awardee  would  be 
selected  based  on  the  best  value  of  a  proven  NDI  capability. 
This  technical  risk  minimization  complemented  the  production 
readiness  assessment  during  which  the  bidders  would  "prove 
out"  their  production  units  in  actual  demonstrations.  The 
production  assessment  would  be  reinforced  by  the  FAT  and  PQT 
(LSDIS  Acquisition  Plan,  p. 11,  1990) . 

Risk  reduction  in  the  LSDIS  acquisition  concentrated  on 
technical  and  cost  risk.  Schedule  risks,  although  classified 
as  moderate,  were  accepted  based  on  the  urgent,  directed 
procurement.  The  limited  quantity  and  urgent  procurement 
classification  allowed  maximum  use  of  waivers  for  all 
nonstatutory  requirements  (LSDIS  Acquisition  Plan,  p. 12, 
1990) . 

The  LSDIS  requirements  risk  evolves  from  the  use  of 
Marine  Corps  requirements .  The  LEWDD  Memorandum  which  served 
as  the  basis  for  the  LSDIS  requirement  was  followed  by  a  short 
market  analysis  but,  like  the  original  GBS  solicitation,  drew 
only  one  bidder,  Lear  Siglar.  With  competition  and  best  value 
common  themes  in  NDI  acquisitions,  the  contract  was  amended  to 
include  a  competitive  procurement  and  interoperability  with 
FAADC3I  at  the  direction  of  the  Army  Acquisition  Executive 
(LSDIS  Quick  Report,  1993) .  These  changes  delayed  the 
solicitation,  added  competition  factors  (which  Lockheed 
Sanders  then  won,  largely  on  the  merit  of  its  proposal  (Eison, 
1994) ) ,  and  entailed  greater  market  analysis  of  contractor 
testing  and  support  capabilities. 

Test  and  evaluation  and  specifications  and  standards 
risks  surface  with  the  emphasis  on  an  urgent  LSDIS  NDI 
procurement.  The  directed  procurement  streamlined  and  waived 
certain  aspects  of  the  program  and  its  documentation.  The 
accelerated  LSDIS  acquisition  program  conducted  performance 
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and  design  testing  after  contract  award  rather  than  during 
candidate  evaluations.  This  did  not  allow  for  verification  of 
system  performance,  production  capabilities,  or  support 
structures  until  after  manufacturing  began.  Once  testing  did 
begin,  the  contractor's  design  did  not  meet  specifications 
during  PQT.  Operational  requirements  were  not  met. 
Integration  and  interoperability  with  FAADC3I  were  also 
overlooked  in  the  requirements.-  Integration  with  the  other 
components  of  FAAD  battalions  emerges  as  a  performance 
specification  issue  as  well.  Market  analysis  with  greater 
depth  may  have  preempted  these  shortfalls. 

The  LSDIS  NDI  acquisition  management  environment  entailed 
greater  risks  than  anticipated.  Test  and  evaluation, 
specifications  and  standards,  ILS,  requirements,  and  NDI 
acquisition  planning  were  streamlined  to  meet  cost  and 
schedule  objectives.  LSDIS  technical  performance  has 
suffered.  Despite  the  streamlining  steps,  which  were  well 
known  to  all  LSDIS  acquisition  participants,  the  contractor's 
product  difficulties  caused  schedule  slippage  and  cost 
increases.  The  short  term  gains  will  be  justified  only  if  the 
amended  portions  of  the  LSDIS  acquisition  cycle  do  not  return 
to  haunt  the  PM.  The  LSDIS  remains  fully  funded  for  40 
sensors  (Department  of  the  Army 'ADA  Highlights  Newsletter,  p. 
April,  1994) 

The  urgent  nature  of  this  NDI  procurement,  similar  to  the 
GBS,  required  more  strict  risk  management.  The  accelerated 
NDI  acquisition  strategy  jeopardizes  the  ease  with  which 
program  objectives  are  expected  to  be  accomplished.  Test  and 
evaluation  and  integrated  logistics  support  plans  may  be 
rushed  to  completion  for  the  sake  of  cost  and  schedule 
objectives.  The  verification  of  performance  and  support 
requirements  in  accordance  with  pre -determined  specifications 
and  standards  may  become  paper  drills  rather  than  evaluations. 
Market  analysis  may  also  be  caught   in  this   compressed 
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acquisition  cycle,  and  critical  performance,  production,  ILS, 
and  program  issues  may  be  overlooked. 

4.   The  FAAD  Sensors  Risk  Management  Plans 

The  FAAD  Sensors  PM  did  not  publish  risk  management  plans 
for  either  GBS  or  LSDIS .  A  risk  management  program  does  not 
imply  the  existence  of  a  explicit  risk  management  plan  (RMP) 
to  outline  the  program.  Risk  planning,  as  required  by  DODD 
5000.1  and  DODI  5000.2,  does  not  require  a  PM  to  release  an 
RMP.  The  decision  is  left  to  the  PM.  Tailoring  a  NDI 
acquisition  strategy  to  meet  program  requirements  allows  the 
PM  to  forego  a  published  RMP.  Tailoring,  however,  leaves  a 
program  vulnerable  to  NDI -specific  risks  in  requirements, 
ILSP,  T&E,  S&S,  and  the  NDI  acquisition  management 
environment.  These  contribute  to  risks  in  its  cost,  schedule, 
performance,  and  supportability  objectives. 

The  absence  of  DOD  or  Army  NDI  risk  management  policy  or 
outlines  leaves  the  burden  on  the  PM.  The  DSMC's  proposed  RMP 
outlined  in  its  Risk  Management  Concepts  and  Guidance 
publication  assumes  a  developmental  program  application.  It 
neglects  the  risks  and  risk  management  needs  unique  to  the  NDI 
acquisition  program.  The  AFAM  likewise  overlooks  the  needs  of 
NDI  PMs  implementing  risk  management  policy. 

Assistance  to  NDI  system  PMs  in  the  form  of  NDI  risk 
management  templates,  sample  NDI  RMPs,  and  specific  DOD  and 
Army  NDI  risk  management  policy  would  facilitate  NDI  risk 
management  practice.  These  measures  would  also  counteract 
PMs'  inclinations  to  treat  risk  management  in  their  NDI 
programs  as  a  costly,  time-consuming,  non- value  added  effort. 
The  desire  to  tailor  and  streamline  developmental  risk 
management  policy  rather  than  work  with  explicit  NDI  risk 
management  guidance  leaves  the  PM  free  game  to  auditors, 
evaluators,  and  stakeholders  in  a  NDI  acquisition  environment 
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that  may  grow  less  forgiving.  The  NDI  templates  and  RMPs 
would  serve  as  readily  available  tools  to  quickly  and 
effectively  assemble  productive  risk  management  programs  that 
successfully  grasp  and  control  NDI  risks.  Improved  DOD  and 
Army  NDI  risk  management  policy  would  reinforce  their 
legitimacy. 

The  ADA'S  FAAD  doctrine  depends  on  EWN.  The  FAAR 
retirement  generated  a  credible  mission  need.  The  need  was 
approved  by  the  Army.  Requirements  were  specified  to  meet  the 
urgent  needs.  The  Gulf  War  highlighted  the  pursuit  of  timely 
system  fielding  (programmatic  risk) .  However,  the  original 
GBS  requirements  were  not  well  written  and  reflected 
insufficient  market  analysis  into  available  early  warning 
radar  technology  (requirements  and  market  analysis  risks) . 

The  GBS  and  LSDIS  NDI  early  warning  radars  provided 
initially  attractive  solutions  to  ADA'S  FAAD  and  force 
readiness  gaps.  Early  operational  assessments,  SSETs,  FATs, 
and  PQTs,  however,  uncovered  weaknesses  in  both  sensor  designs 
that  confirmed  best  value  for  the  Government  but  did  not  yield 
desired  performance  (T&E  and  technical  risk)  .  The  accelerated 
acquisitions  had  rushed  the  "limited"  research  and  development 
process  to  meet  urgent  fielding  requirements  at  the  expense  of 
confirmed,  commercially-validated  requirements  and  performance 
risk  "multipliers"  in  T&E,  S&S,  and  ILS . 

6.   SUMMARY 

The  GBS  and  LSDIS  risks  are  described  by  the  PM  in  terms 
of  cost,  schedule,  performance,  and  supportability. 
Assessments  and  analysis  of  these  risks  are  restricted  to 
outlines  since  no  RMP  is  used  for  either  program.  This  is 
understandable  in  light  of  the  DOD  and  Army  policy  voids  and 
NDI  tailoring  flexibilities.  The  problem  is  that  a  "pick  and 
choose"  policy  environment  frustrates  a  PM' s  definitive 
efforts  to  build  an  effective  risk  management  program  and  to 
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write  an  explicit  RMP.  The  pressure  to  buy  NDI  and  ramrod 
commercial  specifications  and  standards  into  contracts  will 
only  result  in  more  tailoring  and  streamlining  of  RMPs. 
Urgent  and  best  value  labels  put  on  acquisition  programs  will 
continue  the  RMP  and  risk  management  tailoring  trend  in  NDI 
acquisitions.  The  cost  and  time  required  to  staff  RMPs  will 
only  be  justified  when  better  tools,  policies,  and  incentives 
exist  to  prepare  detailed  risk  management  programs. 
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V.   MERGING  THE  TWO  PATHS  INTO  ONE 

A.  PURPOSE 

The  purpose  of  this  chapter  is  to  analyze  DOD  risk 
management  policy  as  it  stands  and  to  propose  changes  that 
support  NDI  acquisition  programs.  The  FAAD  Sensors  risk 
management  lessons  learned  appear  first  in  this  chapter.  The 
importance  of  these  lessons  is  reinforced  by  examining  the 
risk  management  policy  and  the  lack  of  detail  it  provides 
concerning  NDI.  The  chapter  introduces  a  dendritic  diagram 
that  breaks  down  developmental  and  NDI  risk  management 
requirements.  It  then  considers  improvements  to  portions  of 
DOD  risk  management  policy  concerning  NDI  risks.  NDI  risk 
management  tools  are  offered  that  can  help  PMs  in  managing 
risks  in  their  NDI  acquisition  programs.  It  includes  the 
potential  benefits  in  NDI  risk  management  templates,  risk 
management  plans  (RMPs) ,  and  'watch  lists.  It  looks  at 
possible  revisions  to  the  steps  in  the  NDI  acquisition  cycle. 
The  chapter  concludes  with  a  summary  of  where  policy  and 
procedure  changes  will  expedite  and  increase  the  value  of  NDI 
risk  management  programs  and  plans . 

B.  FAAD  SENSORS  LESSONS  LEARNED  IN  RISK  MANAGEMENT 

Several  lessons  can  be  gained  from  the  FAAD  Sensors 
program  in  the  realm  of  risk  management.  The  FAAD  Sensors  PM 
manages  the  GBS  and  LSDIS  NDI  programs  using  DOD's 
developmental  program-based  risk  management  policies.  The 
FAAD  Sensors  program  risks  are  categorized  by  cost,  schedule, 
and  performance  objectives.  Supportability  risks  are  briefly 
mentioned.  Programmatic  risks  are  not  specified.  The  PM  uses 
DODD  5000.1,  DODI  5000.2,  and  DOD  5000. 2 -M  risk  management 
guidance  in  managing  his  NDI  programs . 

In  both  the  GBS  and  LSDIS  programs,  the  Acquisition  Plans 
(APs)  identify  risks,  rate  them,  and  describe  risk  handling 
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and  reduction.  Risk  assessments  are  furnished  in  the 
programs'  Integrated  Program  Summaries  (IPS)  according  to  DOD 
5000. 2 -M  formats.  The  NDI  acquisition  strategy  in  itself  is 
viewed  by  the  PM  as  risk  mitigation.  With  these  risk 
management  program  characteristics  established,  the  following 
lessons  emerge: 

1.  The  FAAD  Sensors  PM  relied  on  developmental  program- 
based  risk  management  policy  direction  and  instruction  because 
no  explicit  DOD  NDI  risk  descriptions,  risk  management 
policies,  or  risk  management  tools  such  as  NDI  risk  templates 
exist. 

2.  The  PM  tailored  and  streamlined  DOD  risk  management 
policy,  risk  assessment  formats,  and  risk  management  tools  to 
meet  the  demands  of  his  NDI  programs'  needs.  While  executing 
his  risk  management  duties  prescribed  by  DOD,  the  PM  is 
vulnerable  to  second-guessing  by  auditors  and  other  critics 
because  specific  NDI  policy  references  that  could  be  useful  to 
justify  risk  management  decisions  and  actions  do  not  exist. 

Tailored  risk  management  practices  and  streamlined  risk 
management  documentation  are  acceptable  in  NDI  DOD  acquisition 
management .  This  is  true  provided  the  PM' s  risk  management 
process  does  not  break  down.  In  an  era  of  constrained 
resources,  smaller  RD&A  budgets,  and  significant  pressures  to 
contain  risks  and  buy  NDIs,  unsuccessful  NDI  risk  management 
programs  spell  program  failure.  More  explicit  NDI  risk 
management  policy  would  protect  programs  if  it  required 
structured  risk  management  planning  in  the  form  of  an  RMP. 
PMs  could  also  protect  the  validity  of  their  risk  management 
efforts  by  using  and  referencing  templates,  program- specif ic 
watch  lists,  risk  models,  or  some  combination  of  these  tools. 
Thus,  the  PM  could  have  a  safety  net  while  walking  the  NDI 
risk  management  high  wire,  with  references  and  justification 
for  risk  ratings,  assessments,  and  handling  measures. 
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3 .  The  GBS  and  LSDIS  IPS  risk  assessments  rate  the 
majority  of  risks  low  while  their  APs  describe  risks  in  ranges 

(generally  low  to  moderate)  .  The  lack  of  consistency  might  be 
related  to  the  difference  in  publication  dates  of  the 
documents  that  guided  these  assessments .  GBS  and  LSDIS  APs 
published  in  1990;  the  GBS  IPS  in  1994;  the  LSDIS  IPS  in  1993. 
In  NDI  program  management  such  as  with  the  FAAD  Sensors  case, 
the  PM  introduces  programatic  risk  by  classifying  more  than  a 
few  program  risk  areas  as  moderate  or  high.  NDI  hold  the 
reputation  of  being  low  risk,  and  to  rate  risks  otherwise  may 
draw  audits,  criticism,  more  documentation  and  meetings,  and 
unwanted  stakeholder  involvement.  Uniformity  of  program  risk 
reviews  prepared  by  contracting  officers  in  APs  that  then 
translates  directly  to  the  PM' s  risk  assessments  in  IPSs 
reduces  the  need  for  such  scrutiny. 

This  uniform  treatment  of  program  risks  and  the  plans  to 
manage  them  could  be  most  appropriately  gained  through  a 
published  RMP  which  serves  as  a  focal  point  for  all  risk 
management  issues  and  actions.  A  published  RMP  can  be  easily 
referenced  by  IPSs  and  APs.  The  PM' s  RMP  could  then  bind  AP 
and  IPS  risk  management  material  together.  This 
decisionmaking  support  tool  could  remain  as  an  available 
program  management  document  from  one  milestone  decision  review 
to  the  next.  Its  proper  use  would  require  training  and 
education  in  NDI  risk  management  planning.  Examples  (see 
Figure  8  and  9)  are  shown  on  the  accompanying  pages. 

4.  The  FAAD  Sensors  PM  does  not  consider  NDI  sources  of 
risk  that  could  potentially  affect  successful  achievement  of 
cost,  schedule,  performance  and  supportability  objectives. 
Requirements,  NDI  Acquisition  Management  Environment,  Test  and 
Evaluation,  Specifications  and  Standards,  and  Integrated 
Logistics  Support  risks  acute  in  NDI  acquisitions  are  not 
recognized  or  anticipated  in  any  GBS  or  LSDIS  IPS  or  AP.  This 
is  because  such  risk  areas  are  only  now  being  identified. 
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Figure  8  Proposed  DSMC  Risk 
Management  Plan  Format  (Source: 
DSMC  Risk  Management  Concepts 
and  Guidance,  Chapter  4,  19  89) 


Although  there  may  be  evidence  of  these  risks  in  the  two 
systems'  test  and  evaluation  master  plans,  systems  engineering 
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master  plan,  and  integrated  logistics  support  plans,  these 
considerations  are  not  cross -walked  with  AP  and  IPS  risk 
documentation . 

These  lessons  are  not  cited  at  the  FAAD  Sensors  PM'  s 
expense.  They  are  learned  with  his  help.  Previous  FAAD 
programs  such  as  the  Sergeant  York,  the  Roland,  and  Air 
Defense  Anti-Tank  System  were  doomed  by  inadequate  risk 
management  programs  whereas  the  GBS  and  LSDIS  programs  are 
consistent  with  what  DOD  risk  management  guidance  requires  in 
print.  The  FAAD  Sensors  NDI  acquisition  approach  is 
consistent  with  the  DOD  SD-2  Buying  NDI  Guide  which  specifies 
market  analysis,  requirements  development,  solicitation, 
source  selection,  product  assurance  (specifications) ,  T&E,  and 
ILS  procedures  (Office  of  the  Secretary  of  Defense  for 
Production  and  Logistics,  1990)  .  Any  trade-offs  made  in 
staffing  a  tailored,  streamlined  NDI  risk  management  program 
versus  other  program  requirements  is  not  the  focus  of  this 
thesis.   However,  the  FAAD  Sensors  case,  like  the  SD-2  and 


75 


other  DSMC  and  DOD  publications',  does  communicate  the  impact 
of  NDI  sources  of  risk  and  their  consequences  in  terms  of 
fielding  delays,  performance  shortfalls,  cost  increases, 
supportability  concerns,  and  programmatic  challenges  (e.g. 
funding  shortfalls  that  reduce  the  number  of  systems  to  be 
fielded,  unfunded  P3I,  the  push  for  CIDs  over  MILSPECS  and 
MILSTDS)  .  The  ADA  user  community  has  as  much  impact  on  the 
control  of  these  risk  events  as  the  PM.  The  PM's  user  has  an 
obligation  to  effectively  research  and  validate  system 
requirements  that  directly  affect  its  divisional  tactical 
doctrine,  force  readiness,  and  potential  combat  capability 
gaps. 

C.   THE  DIVERGING  PATHS 

The  crux  of  the  NDI  risk  management  issue  is  that  no  risk 
management  policy  explicitly  addresses  NDI  acquisitions.  Risk 
management  and  NDI  policy  do  not  converge.  DODD  5000.1  and 
DODI  5000.2  invite  tailoring  of  their  risk  management  policies 
to  fit  the  needs  of  the  individual  programs.  The  implicit  and 
somewhat  erroneous  assumption  in  these  policies  is  that  risk 
management  is  practiced  for  developmental  programs  with 
complete  acquisition  cycles.  NDI  programs  do  not  correspond 
to  these  assumptions.  DOD  policies  neglect  NDI  risks  and  risk 
management  practices. 

NDI  acquisition  risks  are  unique.  They  reflect 
accelerated,  sometimes  urgent  procurements  with  abbreviated 
acquisition  cycles  to  meet  user  needs  and  requirements.  NDI 
acquisitions  tailor  and  streamline  acquisition  policy  and 
particularly,  risk  management  policy,  and  make  tradeoffs  among 
program  objectives.  While  NDI  programs  do  mitigate  risks  in 
cost  and  schedule,  their  technical  performance, 
supportability,  and  programmatic  risks  are  heightened.  The 
consequences  of  risk  events  in  these  areas  can  have  tremendous 
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affect  on  cost  and  schedule  in  the  long  run.  DOD's  challenge 
is  to  bridge  the  gap  between  developmental  acquisition  risk 
management  policy  and  practices  and  those  that  apply- 
specifically  to  NDI  risks.  Effective  application  of  NDI-based 
risk  management  policy  and  practice  can  save  PMs  valuable 
time,  money,  and  manpower  resources  while  assisting  them  in 
effectively  meeting  program  goals  and  objectives. 

Risk  management  programs  and  NDI  solutions  are  popular 
acquisition  issues  whose  paths  must  cross  to  serve  the  best 
interests  of  NDI  PMs,  their  programs,  and  their  customers.  At 
present,  however,  the  two  paths  lack  any  formal  policy 
connection.  The  point  is  not  to  invite  more  regulation,  but 
instead  to  appeal  for  official  documentation  and  guidance  that 
corresponds  with  today's  acquisition  environment  and  its 
emphasis  on  acquiring  state-of-the-art  technologies  today  as 
sources  of  tomorrow's  Army  systems.  By  updating  and 
correcting  the  oversights  in  NDI  risk  management  policy  and 
guidelines,  PMs  can  more  quickly  and  accurately  assemble  risk 
management  programs  and  concentrate  on  program  objectives  with 
an  appropriate  NDI  risk  management  baseline. 

D.   A  DENDRITIC  APPROACH 

DOD  risk  management  policy  exists  in  DODD  5000.1,  page  1- 
4,  and  DODI  5000.2,  Part  5B.  The  Army  published  its 
acquisition  policy  with  references  to  risk  management  in  Army 
Regulation  (AR)  70-1.  Its  Department  of  the  Army  Pamphlet  (DA 
PAM)  70-2  on  Risk  Management  is  due  in  February  1995.  Army 
acquisition  risk  management  policy  implicitly  corresponds  to 
and  mirrors  that  of  DOD.  Army  risk  management  policy 
pertaining  to  NDI  in  AR  70-1  is  not  articulated  despite  the 
popularity  of  NDI  acquisitions  and  Army  NDI  initiatives.  DOD 
risk  management  policy  according  to  DODD  5000.1  and  DODI 


77 


5000.2  relies  officially  on  risk  management  tools  such  as 
templates  outlined  in  DOD  4245. 7-M  and  the  AFAM  and 
unofficially  with  DSMC's  risk  management  plans,  risk  matrices, 
and  watchlists  found  in  its  risk  management  guide.  Army  PMs 
likely  will  tailor  DOD  guidelines  for  implementing  risk 
management  to  suit  individual  program  needs. 

A  model  such  as  a  dendritic  diagram  can  lay  out  the  DOD 
risk  management  issue,  its  scope,  criteria,  measures  of 
effectiveness  and  performance,  and  the  rationale  for  NDI  PMs 
to  have  more  explicit  risk  management  policy.  PMs  could  then 
refer  to  NDI  risk  templates  and  other  tools  to  facilitate 
effective  risk  management. 

Operational  and  design  requirements,  NDI  acquisition 
management,  specifications  and  standards  (for  design  and 
manufacturing  of  standard  and  non-standard  parts) ,  test  and 
evaluation  (including  FATs,  PATs,  and  PQTs)  and  ILS  all 
represent  significant  NDI  risk  areas  lacking  templating  in  the 
DOD  4245. 7-M  and  articulation  in  DSMC  risk  management  plans. 
A  dendritic  layout  of  risk  management  policies  and  the  need 
for  NDI  risk  management  guidance  could  be  outlined  as  seen  on 
the  following  page  in  Figure  10. 

E.   DOD  POLICY  REVISIONS 

NDI  PMs  currently  conduct  risk  identification, 
assessment,  handling,  and  documentation  according  to  DODD 
5000.1  and  DODI  5000.2.  In  doing  so,  the  PM  applies 
developmental  acquisition  risk  management  policy  and 
procedures  focusing  on  traditionally  developmental  program 
risks  to  nondevelopmental  systems.  The  point  is  that,  as  seen 
with  the  FAAD  Sensors  PM,  many  NDI  acquisition  PM' s  are  forced 
to  "think  on  the  move"  without  explicit  directions  or  road 
maps.    Lacking  DOD  or  Army  NDI  risk  management  policy, 
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Figure  10  Dendritic  Diagram  of  DOD  Risk  Management  Policy 
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instructions,  templates,  or  even  unofficial  emphasis  in  DSMC 
risk  management  publications,  PMs  will  continue  to  rely  on  gut 
feelings,  developmental  lessons  learned,  and  incomplete 
references.  The  approach  of  tailoring  and  streamlining 
developmental  risk  management  policy  and  practice  to  suit  NDI 
program  requirements  puts  NDI  PMs  and  their  acquisition 
programs  on  a  high  wire  without  a  net.  Program  risk 
management  can  easily  be  second  guessed  because  no  official 
references  can  be  cited  to  justify  risk  management  decisions. 
Lack  of  published  NDI  RMPs  increases  this  risk  exposure. 

1.   DODD  5000.1  Risk  Management  Policy 

DOD's  Acquisition  Policy  Directive  5000.1,  Part  1 
contains  no  explicit  guidance  directed  to  NDI  risks  and  NDI 
risk  management.  The  implicit  developmental  (as  opposed  to 
NDI)  tone  in  the  risk  management  policy  has  not  kept  up  with 
the  push  to  satisfy  requirements  using  mature  technology 
acquisitions  from  Government  agencies  and/or  commercial 
sources.  This  necessitates  risk  management  program 
requirements  and  references  directed  to  NDI  PMs. 

The  DODD  5000.1  contains  six  areas  which  could  be  revised 
to  make  its  risk  management  guidance  suitable  for  NDI  PMs. 
The  first  area  is  in  the  risk  management  section  description 
in  Part  1.   It  presently  states': 

Program  risks  and  risk  management  plans  shall  be 
explicitly  assessed  at  each  milestone  decision 
point  prior  to  granting  approval  to  proceed  into 
the  next  acquisition  phase. 


This  guidance  lacks  reference  to  a  definitive  risk 
management  planning  structure.  Currently,  NDI  PMs  are 
expected  to  tailor  the  DODD  5000.1  risk  management  guidance  to 
their  NDI  systems  acquisitions.    This  leaves  the  PM's 
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interpretations  of  present  policy  vulnerable  to  second- 
guessing  by  auditors  and  program  stakeholders  such  as  the 
Army.  The  following  could  be  added  for  more  structural 
content  to  developmental  and  NDI  risk  management  planning: 

Every  milestone  decision  point  will  include  a 
review  of  the  updated  risk  management  plan  (RMP) 
and  measures  taken  to  identify,  assess,  analyze, 
handle,  and  document  program  risks.  Continuity 
will  be  maintained  between  the  RMP  and  discussions 
of  program  risks  in  the  Integrated  Program  Summary, 
Acquisition  Plan,  Systems  Engineering  Master  Plan, 
Test  and  Evaluation  Master  Plan,  and  Integrated 
Logistics  Support  Plan. 

The  RMP  would  be  modelled  after  the  DSMC  Risk  Management 
and/or  DSMC  Systems  Engineering  Management  Guides'  formats, 
depicted  in  the  DOD  5000. 2 -M,  and  tailored  to  the  requirements 
of  the  individual  program.  It  would  require  the  "teaming"  of 
the  user  and  the  PM  to  integrate  a  risk  management  focus  with 
needs  and  requirements  development. 

The  second  area  is  subsection  a.  which  describes  critical 
parameters.  This  statement  ignores  the  cost  drivers  in  NDI 
programs:  requirements,  NDI  acquisition  management ,  T&E,  S&S, 
and  ILS.   It  directs  the  following: 

a.    Critical  parameters  that  are  design  cost 
drivers  or  have  significant  impact  on  readiness, 
capability,  and  life  cycle  costs  must  be  identified 
early  and  managed  intensively. 

The  DOD  4245. 7 -M  describes  the  areas  of  risk  that 
jeopardize  successful  cost  objective  achievement.  It  does  not 
describe  NDI  risks  acting  as  cost  drivers.  The  following 
could  be  added  to  reflect  these  NDI  risks: 

NDI  programs  shall  include  the  effects  of 
requirements,  NDI  acquisition  management,  Test  and 
Evaluation,  Specifications  and  Standards,  and 
Integrated  Logistics  Support  risks  on  cost 
objectives.      Measures   of   effectiveness   and 
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performance  for  risk  handling  shall  be  proposed  in 
the  RMP  concerning  these  risks. 

The  third  area  of  interest  is  subsection  c.  which 
discusses  Test  and  Evaluation.   It  reads: 

c.  Test  and  Evaluation  shall  be  used  to  determine 
system  maturity  and  identify  areas  of  technical 
risk. 

NDI  acquisition  strategies  often  include  accelerated  and 
streamlined  (depending  on  the  quantity  and  quality  of 
contractor  test  data)  testing  and  evaluation  processes  or 
simply  contractor  test  and  evaluation  data  reviews.  The  risks 
in  NDI  acquisitions  are  testing  too  little,  too  much,  and  the 
right  or  wrong  requirements.  The  following  should  be  included 
to  address  these  concerns: 

NDI  market  analysis  of  proposed  designs,  NDI  test 
and  evaluation  programs,  and  NDI  performance 
specifications  shall  not  be  subject  to  trade-offs 
for  the  sake  of  short  term  cost  and  schedule 
objectives.  Such  trade-offs  introduce  risks  to  NDI 
lifecycle  costs  and  support.  NDI  contractors'  test 
and  evaluation  data  shall  be  screened  and  validated 
independently  by  the  -operational  test  and 
evaluation  community. 

The  fourth  area  is  subsection  d.  which  considers 
contractor  responsibilities.   It  directs  that: 

d.  Solicitation  documents  shall  require 
contractors  to  identify  risks  and  specific  plans  to 
assess  and  eliminate  risks  or  reduce  them  to 
acceptable  levels. 

The  reference  to  eliminate  risks  could  be  removed  since 
no  program  objective  or  action  is  free  of  risk.  The  shortfall 
in  this  guidance  on  contractor  risk  management  is  the  lack  of 
parallelism  between  the  NDI  PM' s  risk  management  efforts  and 
requirements  and  those  of  the  contractor.    The  NDI  risk 
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management  efforts  should  emphasize  teaming  and  sharing  to 
relieve  the  burden  of  risk  management  from  resting  primarily 
on  the  Government.  This  is  particularly  true  with  the 
accelerated  nature  of  NDI  acquisitions.  Risk  responsibility 
should  be  shared  by  the  contractor.  The  following  could  be 
added : 

Contractor  RMPs  shall  be  specified  in  solicitations 
as  a  deliverable.  These  shall  be  consistent  with 
DOD  program  managers  RMPs .  Risk  sharing  shall  be 
emphasized  by  the  DOD- contractor  team. 

The  fifth  area  focuses  on  subsection  e.  which  discusses 
the  risk  assessment  format  found  in  DOD  5000. 2 -M.  The 
guidance  and  format  make  no  mention  of  NDI  risks.  Risk 
assessments,  while  intended  to  be  tailored,  should  be 
comprehensive  in  nature  and  include  the  full  spectrum  of  risks 
expected  as  a  NDI  PM.  The  five  NDI  risks  should  be  added  to 
the  DOD  5000. 2 -M  format. 

The  final  area  of  revision  is  subsection  f .  which  reads: 

f.  Schedule  shall  be  subject  to  trade-off  as  a 
means  of  keeping  risk  at  acceptable  levels. 

The  FAAD  Sensors  case  demonstrates  that  trading  off 
schedule  to  control  risk  implies  accepting  force  readiness  and 
doctrinal  capability  gaps.  If  no  operationally  effective 
replacement  systems  exist,  then  such  trade-offs  can  have 
significant  impacts  on  force  training  and  combat 
effectiveness.  The  following  clarification  could  be  included: 

Such  trade-offs  shall  be  made  only  in  the  event 
that  current  systems  exist  to  sustain  the  force 
until  the  NDI  are  fielded. 

The  DODD  5000.1  risk  management  policy  revisions  are  an 
important  first  step.  Policy  must  clearly  state  the  risk 
management  requirements  and  guidelines  as  they  specifically 
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affect  a  NDI  PM' s  risk  management  program.  These  changes  will 
provide  impetus  for  revisions  to  DODI  5000.2  which  will  be 
examined  next . 

2.   DODI  5000.2  Risk  Management  Policy 

NDI  PMs  need  more  definitive  and  explicit  policy  and 
procedures  in  DODI  5000.2,  Part  5B  pertaining  to  their  risk 
management  programs.  The  DODI  5000.2,  with  its  developmental 
item  orientation,  neglects  NDI  risk  management.  With 
increasing  competition  for  shrinking  DOD  RD&A  budgets  and 
resources,  DOD  and  Congressional  risk  tolerance  will  decline, 
pressure  to  manage  programs  according  to  sound  business 
practices  will  intensify,  and  "safer"  acquisitions  will 
attract  more  support.  In  these  circumstances,  explicit  NDI 
risk  management  instruction  will  become  critical  to  NDI 
program  management. 

The  two  subsections  in  DODI  5000.2  could  provide  more 
definitive  risk  management  guidance  to  NDI  PM'  s  and  remove  the 
"guesswork"  required  in  tailoring  developmental  risk 
management  policy.  The  first  area  is  subsection  a.  No 
mention  is  made  of  supportability  and  programmatic  risks  in 
the  reference  to  risk  identification  and  control.  These 
objectives  typically  encounter  many  risks  in  NDI  acquisitions 
from  risk  events  concerning  requirements,  NDI  acquisition 
management,  T&E,  and  ILS .  Supportability  and  programmatic 
risks  should  be  included  in  the  discussion  of  acquisition 
program  risks . 

The  DOD  4245. 7-M  templates  referred  to  in  the  policy  as 
a  means  of  identifying  and  controlling  risks  are  ten  years  old 
and  list  risk  areas  such  as  funding  that  are  not  matched  or 
consistent  with  DSMC's  five  objective  areas  of  risk.  The 
templates  have  no  NDI-specif ic ^content .  The  five  areas  of 
NDI  risks  should  be  explicit  in  the  templates  so  that  they 
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correspond  to  NDI  risk  guidance  in  DODI  5000.2.   DOD  4245. 7 -M 
revisions  will  be  discussed  in  the  next  section. 

Subsection  b.  of  DODI  5000.2  addresses  industry 
participation  in  risk  management.  Currently,  there  is  no 
explicit  integration  of  contractor  and  PM  risk  management 
efforts.  A  contractor  RMP  deliverable  similar  to  the  plan 
published  by  NDI  PMs  binds  the  contractor  to  a  risk  management 
partnership  with  the  PM.  It  should  document  and  update  how 
the  program's  risk  identification,  assessment,  analysis,  and 
handling  processes  would  be  conducted.  This  uniform  standard 
would  entail  closer  coordination  or  "teaming"  of  risk 
management  actions. 

3.   DODI  5000.2  Risk  Management  Procedures 

Three  areas  exist  in  DODI  5000.2  risk  management 
implementation  procedures  that  could  better  serve  the 
interests  of  NDI  PM' s  with  more  explicit  discussion  of  NDI 
risks.  First,  the  instruction,  "include  clearly  defined 
criteria  for  elements  leading  to  the  risk  assessment  events. . . 
satisfaction  of  these  criteria  must  be  well  documented  to 
support  the  rigor  necessary  in  the  risk  assessment  process" 
highlights  the  policy  void  NDI  PMs  face  in  justifying  their 
risk  ratings  and  assessments.  DOD  4245. 7 -M  serves  as  the 
origin  for  the  criteria  described.  Without  existing  NDI  risk 
criteria,  risk  assessments  have  no  basis  or  references  to 
support  them. 

The  second  area  of  this  procedures  section  concerns  the 
risk  assessment  format  in  DOD  5000. 2 -M.  The  format 
description  does  more  to  encourage  tailoring  than  to  depict  a 
specific  risk  assessment  format;  it  provides  little  to  support 
outlining  an  NDI  risk  assessment.  The  format  would  be  more 
consistent  if  it  were  to  correspond  to  the  risk  areas  listed 
in  the  DOD  4245. 7 -M.   Additionally,  with  NDI  and  streamlining 
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initiatives  more  common  today,  more  detail  is  required  in  the 
supportability  and  programmatic  sources  of  risk  currently 
described  in  those  sets  of  templates. 

Finally,  milestone  decision  point  reviews  discussed  in 
this  portion  of  DODI  5000.2  do  not  refer  to  one  central 
document  as  the  basis  for  risk  management  data  and 
documentation  of  the  planned  risk  management  program. 
Reference  should  be  made  to  a  RMP  as  the  source  of  documented 
program  risks,  risk  assessments  and  analysis,  risk  reduction 
measures,  rationale,  and  assumptions  in  the  published  risk 
ratings.  This  could  be  more  effective  than  flipping  between 
the  IPS,  AP,  TEMP,  and  SEMP. 

Alternative  acquisition  strategies,  particularly  NDI 
acquisition  strategies,  are  often  designed  to  mitigate  certain 
risks.  NDI  acquisition  strategies  account  for  degrees  of  risk 
since  they  are  based  on  procuring  mature  technology  with 
abbreviated  RD&A  schedules  and  costs.  The  policy  here  should 
explicitly  cite  the  NDI  alternative,  its  streamlined 
acquisition  cycle,  and  the  parts  of  DODD  5000.1  and  DODI 
5000.2  in  which  detailed  descriptions  of  NDI  risks  and  risk 
management  actions  appear.  For  example,  the  following  parts 
of  DODI  5000.2  could  feature  the  NDI  risks,  possible  risk 
ratings  and  assessment  considerations,  and  initial  risk 
handling  options  as  deliverables:  Part  4  (Requirements),  Part 
6  (Test  and  Evaluation) ,  Part  7  (Integrated  Logistics 
Support) ,  Part  9  (Specifications  and  Standards) ,  and  Part  11 
(NDI  Acquisition  Management) .  Risks,  particularly  NDI  risks, 
could  then  be  distinguished  from  developmental  features  in 
each  of  these  parts  of  the  DODI  5000.2. 

F.   DOD  4245. 7 -M  TEMPLATES  IMPROVEMENTS 

The  following  are  areas  for  revision  in  DOD's  primary 
risk  management  tool,  the  risk  templates.  This  would  clarify 
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DOD's  position  on  NDI  risks  and  recommended  risk  management 
measures  to  the  benefit  of  NDI  PMs,  their  programs,  and  their 
customers. 

1.   Requirements  Risks 

NDI  risk  management  templates  should  begin  with 
requirements  risk.  Requirements  risk  should  be  identified  and 
assessed  first,  prior  to  considering  a  program's  risks  to  its 
cost,  schedule,  technical  performance,  supportability,  and 
programmatic  objectives.  Identifying  and  assessing 
requirements  risks  controls  its  potential  impact  on  technical 
performance,  cost,  schedule,  and  performance  specifications 
from  stalling  an  NDI  program's  early  progress.  NDI 
requirements  must  include  integration  and  interoperability  of 
the  NDI  with  developmental  items  and  the  corresponding  risks 
of  failing  to  effectively  integrate  and  interface.  An  example 
would  be  the  risk  of  LSDIS  not  having  interoperability  with 
FAAD  command  and  control  nodes  and  fire  units  such  as  Avenger 
because  that  requirement  was  not  concretely  established. 

Inserting  market  analysis  into  the  acquisition  cycle  and 
screening  requirements  development  against  what  actually 
exists  in  the  market  would  limit  unrealistic  requirements 
appearing  in  RFPs .  The  market  analysis  step  should  be 
specified  in  policy  as  a  step  initially  occurring  between 
preparation  of  the  MNS  and  the  ORD.  This  would  facilitate  NDI 
solutions  to  user  requirements  and  support  sound  NDI 
acquisition  strategy  development  based  on  a  clear  picture  of 
current  industry  technology  rather  than  nonexistent,  desired 
capabilities  of  multiple  systems  blended  together.  The 
category  of  NDI  would  also  be  clearly  identified  such  that 
source  selection  evaluation  test  results  concerning  assessment 
of  technical  performance  would  not  necessitate  redesign  and 
recompetition  of  requirements  and  potential  designs. 
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2.   Requirements  Risk  Template 

AREAS  OF  RISK: 

Operational  and  Design  Requirements  that  are  ill -defined 
or  overlook  NDI  alternatives. 

Inadequate  Market  Analysis  that  contributes  to  "mix  and 
match  technologies  being  required  rather  than  verification  of 
what  technology  actually  exists. 

Tradeoffs  in  minimum  and  objective  performance 
requirements  that  fail  to  meet  the  user's  stated  need. 

Improperly  defining  the  proposed  system's  NDI  category 
and  required  design  modifications. 

OUTLINE  FOR  REDUCING  RISK; 

1.  Insert  NDI  Market  Analysis  into  the  acquisition  cycle 
as  part  of  Phase  0,  Concept  Exploration  and 
Development. 

2 .  Apply  PM  -  user  teaming  to  better  screen  and  develop 
requirements . 

TIMELINE :  All  Phases 

3.  Specifications  and  Standards  Risks 

NDI  risk  management  policy  should  emphasize  which 
specifications  and  standards,  CIDs  or  MILSPECS,  that  NDI 
procurements  will  comply  with.  Secretary  of  Defense  William 
Perry  is  on  record  as  requiring  the  use  of  commercial 
specifications  and  standards  rather  than  MILSPECS  and  MILSTDS 
(Secretary  of  Defense  Memorandum  on  Specifications  and 
Standards,  p.  1,  1994) : 

Use  performance  and  commercial  specifications 
and  standards  in  lieu  of  military  specifications 
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and  standards,  unless  no  practical  alternative 
exists  to  meet  the  user's  needs. 


The  DODI  5000.2  clearly  states  the  waiver  required  to  use 
MILSPECS  and  MILSTDS  over  CIDs  found  in  the  Uniform  Commercial 
Code.  DOD  policy  on  technical  risk  reduction  by  using  mature 
NDI  systems  and  their  corresponding  mature  commercial 
specifications  and  standards  can  assist  PM'  s  and  simplify 
contractor  compliance.  Rather  than  specifications  that 
specify  system,  design,  and  manufacturing  functions  and  steps, 
the  use  of  performance  specs  based  on  form,  fit,  and  function 
in  place  of  the  current  "how-to"  specs  better  suits  mature 
technology  found  in  NDI.  Such  a  change  would  facilitate 
contractor  responsiveness  to  RFP  requirements  and  draw  more 
competitors  to  the  bidding  process.  A  larger  sample  of 
industry  alternatives,  as  witnessed  in  the  re- competition  of 
the  GBS  contract  and  validated  in  SSETs,  encourages 
competitive  systems  and  prices.  Using  CIDs  would  replace  the 
red  tape  NDI  systems  face  in  complying  with  developmental 
MILSPECS  and  MILSTDS. 

4.   Specifications  and  Standards  Risk  Template 

AREAS  OF  RISK: 

Technical  performance  in  commercial  applications  as 
specified  in  CIDs  may  not  equate  to  or  explicitly  meet 
technical  performance  in  military  applications  as  stated  in 
MILSPECS  and  MILSTDS  after  an  NDI  acquisition  strategy  is 
already  approved. 

Performance  S&S  based  on  form,  fit,  and  function  (that 
allow  contractors  to  design  solutions)  instead  of  the  "how-to" 
MILSPECS  and  MILSTDS  used  in  design  and  manufacturing  may 
encounter  workforce  resistance  or  complacency. 

Inadequate  market  analysis  leads  to  accepting  products 
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with  insufficient  or  undocumented  technical  data  or  CIDs  with 
which  to  re-compete  the  procurement  for  future  buys. 
OUTLINE  FOR  REDUCING  RISKS; 

1.  Specify  in  solicitations  that  CIDs  for  meeting  user 
requirements  are  a  deliverable. 


2.  Evaluate  the  CIDs  against  MILSPECS  to  verify  their 
adequacy  for  design  and  development. 


3 .  Train  and  educate  the  acquisition  workforce  in  CIDs 
and  commercial  specifications. 


TIMELINE:   All  Phases 

5.   NDI  Acquisition  Management 

NDI  acquisition  management  risks  impact  programmatic, 
cost,  and  schedule  objectives.  NDI  risk  management  policy 
should  redirect  risk  management  responsibilities  to  be  borne 
primarily  by  industry,  or  at  least  equally  shared.  Since  NDI 
is  mature  technology,  the  developmental  program  risks  assumed 
by  the  PM  and  the  Government  no  longer  apply.  With  CIDs  on 
hand  and  effective  market  analysis  verifying  qualified 
contract  bidders,  the  RFP  and  SSET  processes  become  much 
simplified. 

Changing  developmental  program  paradigms  to  NDI  remains 
an  acquisition  community  challenge.  NDI  alternatives  can 
become  more  acceptable  to  the  acquisition  workforce  with 
revised  risk  templates  and  detailed  articulation  of  NDI  risks. 
Streamlining  risk  management  paperwork  requirements  would  be 
best  instituted  by  using  RMPs  and  improved  education.  In 
addition,  delegating  reviews  of  program  risks  to  lower  levels 
of   the  acquisition  chain  of   command  would  speed  risk 
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management  actions. 

The  integration  of  civilian  technologies  into  the 
military  demands  that  contractor  risk  management  programs  and 
plans  be  explicitly  required  items  in  RFPs .  NDI  acquisition 
management  requirements  mandates  parallel  NDI  risk  management 
policy  and  references  for  both  industry  and  PMs . 

6.   NDI  Acquisition  Management  Risk  Template 

AREAS  OF  RISK: 

Despite  streamlining,  paperwork  requirements,  pricing 
data,  accounting  requirements  and  continuous  audits  of  NDI 
programs  stifle  the  cost  and  schedule  objectives  laid  out  in 
the  acquisition  strategy. 

Traditional  developmental  program  attitudes  and 
developmental  program  "mindsets"  continue  to  reflect  a 
cultural  resistance  toward  implementing  timely  and  cost- 
effective  NDI  acquisition  strategies. 

Lack  of  PM- contractor  and  user-PM  teaming  on  risk 
responsibility  and  risk  sharing  hampers  flexibility  in  risk 
management  efforts. 

Programmatic  micromanagement  by  stakeholders  defeats  the 
benefits  of  an  NDI  acquisition  strategy. 

OUTLINE  FOR  REDUCING  RISKS: 

1.  Require  RMPs  as  a  contractor  deliverable. 


2.  Require  workforce  training  and  education  in  NDI 
through  DSMC. 


TIMELINE:  All  Phases 


91 


7.  Test  and  Evaluation 

NDI  Test  and  Evaluation  risks  impact  performance,  cost, 
schedule,  and  supportability  objectives.  NDI  risk  management 
policy  must  address  the  accelerated  T&E  program  for  validating 
component  and  system  performance  in  operational  testing.  An 
NDI  evaluation  would  be  expedited  with  a  test-f ix-retest 
(FAADS  PMO,  1994)  cycle  rather  than  the  extensive 
documentation  that  goes  into  describing  all  the  system  faults 
prior  to  a  rescheduling  a  test.  Technical  performance  and 
design  risks  as  well  as  confirmation  of  compliance  with 
realistic  specifications  and  standards,  requires  more  detailed 
testing  of  the  product  and  production  process  than  implied  in 
the  NDI  mature  technology  "label". 

8.  Test  and  Evaluation  Risk  Template 

AREAS  OF  RISK; 

Requirements  are  not  stable,  realistic,  or  well- 
understood  by  designers,  developers,  testers,  or  managers. 

Overtesting  conducted  despite  the  presence  of 
satisfactory  contractor  test  and  evaluation  data  package. 

Developmental/technical  testing  costs  are  saved  but 
operational  testing  for  operational  effectiveness  and 
suitability  may  involve  conditions  not  grasped  by  contractor 
testing  program.  These  incomplete  tests  and/or  data  may  be 
overlooked  or  unquestioned  in  the  accelerated  NDI  acquisition 
cycle  and  corresponding  accelerated  NDI  testing  program. 

OUTLINE  FOR  REDUCING  RISKS: 

1.  Test  and  evaluation  data  reviews   of   contractor 
commercial  testing  program  and  results. 
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2.  Demonstrations  of  the  contractor's  testing  process. 


3.  Test-Fix-Retest   the  proposed  design  on   site   to 
expedite  testing. 


TIMELINE :  All  Phases 

9.   Integrated  Logistics  Support 

NDI  ILS  risks  affect  cost,  schedule,  performance,  and 
supportability  objectives.  NDI  risk  management  policy  on 
logistics  planning  risks  entails  the  greatest  need  in  improved 
DOD  guidance.  The  supportability  risk  of  contractor 
production  and  support  operations  terminating  prior  to  a  time 
convenient  to  the  Government,  mandates  that  options  be 
specified  in  the  ILSP  and  agreed  upon  by  the  PM  and  the 
contractor. 

Options  the  PM  can  consider  include  those  posed  in  DSMC's 
NDI  Acquisition  publication: 

a.  Buy  commercial  upgrades  as  they  evolve  and 
become  available. 

b.  Make  a  one-time  mass  spares  purchase  to  sustain 
the  duration  of  the  system's  lifecycle. 

c.  Buy  the  technical  data  package  to  solicit 
sources  of  supply  that  coincide  with  the  end  of  original 
production  and  support  by  the  original  contractor. 

Market  analysis  should  reveal  whether  contractor 
technical  data  rights  conflict  with  Government  interests  and 
whether  longevity  of  profitable  production  lines  pose  risks  to 
achieving  supportability  objectives.  These  should  all  be 
points  of  direction  in  DOD  policy. 
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10.   Integrated  Logistics  Support  Risk  Template 

AREAS  OF  RISK: 

Technical  Data  Packages  may  be  unavailable  or  incomplete 
which  creates  instability  of  spares  and  parts  access. 

Competitive  re -procurements  of  parts  may  not  contain 
proper  incentives  to  attract  spares  and  parts  vendors. 

ILS  and  lifecycle  focus  may  be  overlooked  during  the 
requirements  development  stage. 

Use  of  military  standard  and  non-standard  parts  creates 
multiple  parts  and  spares  lines. 

Depot/repair  levels  may  not  be  defined  in  terms  of 
operational  environments. 

OUTLINE  FOR  REDUCING  RISKS: 

1.  Define  ILS  requirements  when  deciding  what  category  of 
NDI  the  acquisition  strategy  involves. 


2 .  Conduct  market  analysis  of  contractor  ILS 
capabilities,  ILS  testing  and  support  demonstrations 
in  the  intended  operational  environment  and 
conditions. 


3.  Specify  training  packages  and  publications  as  a 
contractor  deliverable. 


G.   SUMMARY 

The  NDI  areas  described  above  require  timely  DOD 
attention  in  DODD  5000.1,  DODI  5000.2,  and  DOD  4245. 7-M. 
These  revisions  are  needed  to  adequately  provide  guidance  to 
NDI  PMs .  The  template  diagram, .the  specific  area  of  risk,  an 
outline  for  reducing  the  risk,  and  a  timeline  for  managing 
these  risks  within  the  system's  lifecycle  must  be  included. 
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Clarifying  the  risk  management  policy  and  corresponding 
templates  with  NDI  implications  would  streamline  the  risk 
management  process  in  itself  by  saving  time  and  effort  to 
staff  who  manage  a  risk  management  program.  There  would  be  no 
need  for  NDI  interpretation  of  developmental  item  templates. 
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VI.   SUMMARY,  CONCLUSIONS,  AND  RECOMMENDATIONS 

A.  SUMMARY 

The  subjects  of  risk  management  and  NDI  acquisitions  are 
attracting  more  attention  today.  The  individual  emphasis  on 
these  two  acquisition  management  areas  is  not  matched  by  their 
effective  integration.  As  a  result,  NDI  PMs  are  conducting 
risk  management  programs  that  are  themselves  at  risk  due  to 
the  lack  of  current,  explicit  guidance  from  DOD. 

The  practice  of  risk  management  does  not  benefit  from 
"cookbook"  solutions.  If  such  solutions  existed  for 
developmental  acquisition  programs,  there  are  very  few,  if 
any,  for  NDI  acquisition  programs.  This  is  the  NDI  PM' s 
dilemma.  The  NDI  PMs  must  adhere  to  DODD  5000.1  and  DODI 
5000.2  risk  management  policies  in  their  programs,  but  the  DOD 
material  is  vague  or  must  be  tailored  to  support  NDI  program 
needs .  The  FAAD  Sensors  case  is  a  testament  to  the  PM  who 
drives  a  risk  management  program  without  the  benefit  of 
signposts  or  road  maps. 

The  NDI  PM's  challenges  and  program  risks  in  the  mid- 
1990 's  are  not  adequately  reflected  in  DOD's  mid- 1980 's  risk 
management  policy,  procedures,  or  tools.  Risk  management  will 
receive  more,  not  less,  emphasis  as  an  explicit  management 
function.  NDI  and  NDI  acquisition  strategies  will  continue  to 
grow  in  popular  support  as  DOD  RD&A  budgets  "downsize". 
Modernization  at  minimum  risk  therefore  will  require  properly 
marked  signposts  and  a  good  road  map.  It  is  time  for  DOD  to 
print  and  distribute  those  signs  and  maps. 

B.  CONCLUSIONS 

NDI  PMs  continuously  manage  risk  as  part  of  today's 
streamlined  and  tailored  NDI  acquisition  environment.  They 
should  expect  and  receive  succinct,  explicit  policy  and 
guidelines  to  help  them  meet  their  risk  management  and  program 
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management  goals  and  objectives.  DODD  5000.1  could  better 
serve  the  NDI  PMs  with  risk  management  language  directed  to 
their  specific  type  of  programs  and  acquisition  strategies. 
DODI  5000.2  could  provide  NDI  PMs  with  both  better  risk 
management  guidance  and  implementation  procedures  with  NDI- 
based  instructions,  formats  (to  be  found  in  DOD  5000. 2 -M)  ;  and 
tools  (DOD  4245.  7-M  risk  templates  and  risk  management  plan)  . 

DOD  4245. 7-M  could  introduce  NDI  as  a  topic  and  an  area 
of  templating  as  it  is  updated  for  today's  acquisition 
environment.  The  risks  uniquely  significant  to  NDI 
acquisition  programs  can  be  better  recognized,  understood,  and 
managed  with  a  focal  point  known  as  a  risk  management  plan. 

Several  common  threads  bind  NDI  and  NDI  risk  management 
to  immediate  programmatic,  performance,  and  supportability 
objectives  and  long-term  cost  and  schedule  objectives.  Those 
threads  are  thorough  market  analysis,  accelerated  acquisition 
cycles,  and  teaming  of  the  risk  management  effort  between  the 
PM  and  the  contractor  as  well  as  the  PM  and  the  user.  The 
acquisition  environment  and  its  intricacies  may  be  changing 
but  the  demand  for  short -duration,  cost-effective  systems 
acquisitions  will  persevere. 

C .   RECOMMENDATIONS 

DOD  should  upgrade  the  DODD  5000.1,  DODI  5000.2,  DOD 
4245. 7-M,  and  DOD  5000. 2 -M  to  incorporate  the  NDI  risk 
management  issues  and  proposed  revisions  discussed  in  this 
thesis.  DSMC  publications  referring  to  risk  management  and 
the  AFAM  should  be  similarly  revised. 

Market  analysis,  risk  management  plans,  teaming,  and  NDI 
acquisition  training  and  education  should  be  explicitly 
emphasized  in  DOD  risk  management  policy.  Market  analysis 
should  be  specified  as  a  deliberate  step  or  phase  conducted 
immediately  after  the  release  of  the  Mission  Needs  Statement. 
This  may  coincide  with  the  Concept  Exploration  and  Development 
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Phase .  Risk  management  plans  should  be  a  required  program 
management  and  decisionmaking  tool  that  is  published  as  the 
primary  source  of  risk  management  program  information  such  as : 
risks,  risk  ratings,  risk  assessments,  risk  analysis,  risk 
handling  and/or  reduction  measures,  and  risk  data 
documentation.  Other  program  plans  would  refer  to  the  RMP  as 
the  consolidated  reference  point  for  program  risk  information. 
Teaming  and  education  on  NDI  risk  management  should  be 
directives  required  of  all  PMOs . 

Risks  in  the  following  NDI  sources  of  risk  should  be 
explicitly  cited  and  referenced  in  DOD  risk  management  policy: 
Requirements,  Test  and  Evaluation,  Specifications  and 
Standards,  Integrated  Logistics  Support,  and  the  NDI 
Acquisition  Management  Environment.  These  sources  create 
uncertainty  in  the  successful  accomplishment  of  NDI  program 
objectives.  They  should  be  explored  from  both  a  NDI  and  a 
developmental  perspective  (based  on  the  needs  and  requirements 
of  the  user) . 

DOD  NDI  risk  management  policy  should  stress  RMPs,  CIDs 
(for  review  and  comparison  to  MILSPECS) ,  Test  and  Evaluation 
program  data  packages  and  demonstrations,  and  Integrated 
Logistics  Support  Plans  in  NDI  RFPs  as  contractor 
deliverables.  These  should  be  required  up  front  and  early  in 
the  accelerated  NDI  acquisition  cycle  and  subject  to  frequent 
review  based  on  their  immediate  and  long-term  lifecycle 
impacts  on  the  system  being  acquired. 

D.   AREAS  FOR  FURTHER  RESEARCH 

1.  Market  Analysis  of  NDI  Requirements.  The  importance 
of  market  analysis  grows  as  DOD  turns  to  the  commercial 
marketplace  for  more  immediate  solutions  to  its  needs.  An 
analytical  approach  to  evaluating  NDI  solutions  is  needed  to 
replace  the  market  surveys  of  the  past.  This  is  more  relevant 
given  the  close  ties  and  tremendous  impacts  well-  or  poorly- 
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done  market  analysis  can  have  on  requirements  generation, 
selection  of  specifications  and  standards  based  on  those 
requirements,  and  comparison  of  contractor  test  and  evaluation 
results  with  those  sought  through  DOD's  developmental  and 
operational  testing  programs.  Market  analysis  also  plays  a 
significant  role  in  determining  contractors'  ability  to 
support  their  products  throughout  their  lifecycles.  A  case 
study  of  NDI  market  analysis  success  story  would  prove 
valuable  as  a  reference  or  model  to  all  acquisition  community 
players . 

2.  Successful  Contractor  RMPs  (in  the  Context  of  Best 
Practices) .  The  case  analysis  of  one  or  several  success 
stories  in  contractor  risk  management  bears  investigation, 
particularly  in  the  NDI  environment.  The  lack  of  DOD  emphasis 
on  risk  management  of  NDI  acquisition  programs  and  supporting 
RMPs  may  be  mirrored  by  a  lack  of  commercial  emphasis  on  the 
same  area.  Documentation  of  lessons  learned  in  contractor 
risk  management  plans  could  offer  valuable  insights  to  the 
user  community,  PMs,  and  other  contractors.  Providing  a 
contractor  "best  practices  risk  management  model"  could 
reinforce  the  importance  placed  on  contractor  RMPs  as 
deliverables. 

3 .  Risk  Management  in  Foreign  Military  Sales  and 
Foreign  Military  NDI  acquisition  programs.  The  risks 
encountered  with  acquiring  NDI  may  represent  only  one  category 
of  emerging  acquisition  issues  facing  DOD.  Risks  and  risk 
management  practices  in  foreign  procurements  may  have 
similarities  to  NDI  acquisition  programs.  This  is  especially 
true  with  our  national  security  strategy's  increased  focus  on 
economic  security  and  global  competitiveness.  The  DTIB's 
vulnerability  to  not  only  national  NDI  alternatives  but 
international  NDI  may  provide  more  justification  for  dual -use 
products  to  improve  competitiveness  and  mitigate  RD&A  risks. 
The  risks  and  risk  management  practiced  in  foreign  acquisition 
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programs  will  become  more  complex  and  may  require  more 
detailed  DOD  risk  management  policy  in  foreign  technology 
acquisition  programs. 

4 .  The  DOD  Developmental  Acquisition  Program  Community's 
Ability  to  Cope  with  the  Rising  Tide  of  NDI  and  Commercial 
Practices.  The  cultural  biases  and  mindsets  towards 
developmental  programs  described  in  this  thesis  must  face  the 
rising  tide  of  NDI  much  like  the  DTIB  faces  increased 
competition  for  shrinking  RD&A  dollars.  Changing  attitudes 
and  program  management  paradigms  in  the  acquisition  community 
has  tremendous  time,  resource,  and  personnel  implications. 
Streamlining  and  tailoring  the  DOD  acquisition  process 
emphasizes  immediate,  mature,  and  cost-effective  NDI 
alternatives.  It  may  be  important  to  examine  the  training  and 
education  programs  being  used  to  facilitate  NDI  acquisitions, 
NDI  risk  management,  and  acquisition  community  responsiveness 
to  a  changing  acquisition  environment. 
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